Abstract:
Embodiments of the invention provide a device and a method for automatically testing embedded software, and more specifically for testing interfaces between layers of the embedded software. In one embodiment, the device includes: an emulator; a server including embedded software; an evaluation board configured to download the embedded software from the server and controlled by the emulator; and a host system configured to receive the embedded software from the server and automatically generate test cases for testing the embedded software using the emulator.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This U.S. non-provisional patent application claims priority under 35 U.S.C. §119 of Korean Patent Application No. 10-2007-0040128, filed on Apr. 25, 2007, the entire contents of which are hereby incorporated by reference. 
       BACKGROUND OF THE INVENTION 
       [0002]    1. Field of the Invention 
         [0003]    The present invention disclosed herein relates to an embedded software test, and more particularly but without limitation, to a device and a method for automatically testing interlayer interfaces of the embedded software. 
         [0004]    2. Description of the Related Art 
         [0005]    A typical tool for testing embedded software is a debugger equipped with a part of integrated development environment. Other test tools are limited to analyzing the performance of embedded software. 
         [0006]    A debugger generally provides functions such as break points, symbol monitoring, and stack monitoring. For example, Microsoft&#39;s Visual Studio .NET software development tool can set a break point at any point in a code listing, monitor and change global and local variables, and read and modify memory regions in code and data sections. However, these debugging functions are performed in an ad-hoc manner according to the personal experience and expertise of a software engineer or other tool users. A more objective and systematic approach is needed. 
         [0007]    A typical test tool for analyzing embedded software performance generally provides functions such as memory usage analysis, and execution path analysis. For instance, IBM&#39;s “Rational Test RealTime” and Metrowerks&#39; “CodeTEST” use software analysis functions within the framework of a system test after regarding the embedded software as a single functional unit. However, embedded software performance is closely related to the interaction between software and hardware components such as between an operating system (OS) and device drivers. For this reason, integration tests between such components are very important, and a method for supporting such integration testing is needed. 
       SUMMARY OF THE INVENTION 
       [0008]    The invention provides a device and method for automatically testing interlayer interfaces in embedded software. 
         [0009]    An embodiment of the invention provides a method for testing embedded software including: downloading the embedded software; analyzing the embedded software to identify an interlayer interface in the embedded software; associating the identified interlayer interface with one of a plurality of predetermined interface types; and generating test cases for the identified interlayer interface based at least in part on the one of the plurality of predetermined interface types. 
         [0010]    Another embodiment of the invention provides a processor-readable storage medium, the processor-readable storage medium having stored thereon processor-executable code, the processor-executable code adapted to perform the foregoing method when executed by the processor. 
     
    
     
       BRIEF DESCRIPTION OF THE FIGURES 
         [0011]    The accompanying figures are included to provide a further understanding of the present invention, and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the present invention and, together with the description, serve to explain principles of the present invention. In the figures: 
           [0012]      FIG. 1  is a block diagram of a test system according to an embodiment of the present invention; 
           [0013]      FIG. 2  is a block diagram of an interface of an embedded system according to an embodiment of the present invention; 
           [0014]      FIG. 3  is an illustration of interfaces between components in  FIG. 2 ; and 
           [0015]      FIGS. 4A and 4B  are flowcharts of a test operation order according to an embodiment of the present invention. 
       
    
    
     DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
       [0016]    Preferred embodiments of the present invention will be described below in more detail with reference to the accompanying drawings. The present invention may, however, be embodied in different forms and should not be constructed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present invention to those skilled in the art. 
         [0017]    Embodiments of the invention disclosed herein relate to a method for automatically testing embedded software in an emulator environment. 
         [0018]      FIG. 1  is a block diagram of a test system  100  according to an embodiment of the present invention. 
         [0019]    Referring to  FIG. 1 , the test system  100  includes a host system  10 , an emulator  20 , an evaluation board  30 , and an OS server  40 . 
         [0020]    The host system  10  generates test cases for testing an embedded software interface. The emulator  20  executes the test case generated from the host system  10 , and then outputs the result. The evaluation board  30  is provided to test the embedded software  41  in hardware. The OS server  40  includes the embedded software  41 , and loads the embedded software  41  into the evaluation board  30  under the control of the emulator  20 . 
         [0021]    The host system  10  includes an embedded software analyzing engine  11 , a test case generating engine  12 , a test executing engine  13 , and a test result analyzing engine  14 . The embedded software analyzing engine  11  receives source code or object code of the embedded software  41  from the OS server  40 . The embedded software analyzing engine  11  automatically identifies the specific location from an executable linker format (ELF) file; each of the locations represents where each interface symbol corresponding to each interface of the embedded software  41  is mapped in the source code or the object code. The test case generating engine  12  generates test cases for testing the embedded software  41 . The test case is written in a script language, which can be executed on the evaluation board. The test executing engine  13  provides the test case from the test case generating engine  12  to the emulator  20 , and controls the emulator  20  for execution. The test result analyzing engine  14  receives a test result from the emulator  20 , and analyzes the received test result. The test result analyzing engine  14  reports an analysis of the test result. The analysis may include, for instance, a summary of test coverage, fault information, and performance data. 
         [0022]      FIG. 2  is a block diagram of an interface of an embedded system according to an embodiment of the present invention. 
         [0023]    Referring to  FIG. 2 , the embedded system includes a hardware layer  31 , a hardware abstraction layer (HAL)  32 , an operating system (OS) layer  35 , and an application layer  36 . The OS layer  35  includes embedded software  33  and kernel software  34 . As shown, device drivers are connected to the OS layer  35  and the HAL  32 , and are embodied in software units. 
         [0024]    The embedded software  33  includes software units (SU&#39;s) in multiple layers, which are organically connected to each other. The software units in each layer are divided into kernel-dependent software unit (SUk), device-dependent software unit (SUd), and processor-dependent software unit (SUp) types. The SUk depends on kernel software, and is an OS service like an application program interface (API). The SUd is software for supporting a physical device such as a LCD controller. The SUp is HAL software for hardware device initialization and configuration, and depends on a target processor. 
         [0025]      FIG. 2  also illustrates hardware units HUa 1  and HUa 2  included in hardware layer  31 . HUa 1  is associated with a register and Random Access Memory (RAM); HUa 2  is associated with a timer, a bus, or a Universal Asynchronous Receiver/Transmitter (UART). 
         [0026]      FIG. 3  is an illustration of interfaces between components in  FIG. 2 . 
         [0027]    Referring to  FIG. 3 , an interface associated with a hardware unit (HU) that can be directly or indirectly accessed from embedded software is defined as a hardware part interface (HPI). An interface with respect to an OS part is called an OS part interface (OPI). In total,  FIG. 3  illustrates five types of interfaces. 
         [0028]    First, a hardware part interface HPI 1  exists when a software unit such as SUd or SUp directly accesses a hardware unit HUa 1  (such as RAM capable of reading/writing data). 
         [0029]    Second, a hardware part interface HPI 2  exists when a software unit such as SUp indirectly controls a hardware unit HUa 2  (such as a timer) through an address of a HUa 1  register. 
         [0030]    Third, an OS part interface OPI 1  exists when the SUd and SUp call functions of SUk that are not related to the control of hardware unit. In this instance, the called functions of SUk could include, for example, task management, inter-task communication, system calls related to an exceptional handling, and certain API functions. 
         [0031]    Fourth, an OS part interface OPI 2  exists when the SUd and SUp call functions of SUk that are directly related to a hardware unit HUa 1 . In this case, the called functions of SUk could include system calls for managing a virtual memory. 
         [0032]    Fifth, an OS part interface OPI 3  exists when the SUd and SUp call functions of SUk that is indirectly related to a hardware unit (e.g., where a HUa 2  is indirectly controlled through a HUa 1 ). The called functions of SUk may relate, for example, to physical memory management, time management, interrupt handling, IO management, networking. 
         [0033]    Each of the five interface types above may be associated with a test interface symbol used to generate the test case, as described below with reference to  FIGS. 4A and 4B . 
         [0034]    An embodiment of the present invention relates to an automation device and method for testing an embedded software interface. Such automation features are integrated into an emulator that includes basic debugging and performance monitoring functions necessary to test embedded software interfaces. An embedded software test automation method of the present invention automatically identifies an interface, generates test cases according to an interface test criteria, and performs a test on embedded software. Moreover, debugging processes such as fault detection and fault cause search are integrated into the test process. 
         [0035]      FIGS. 4A and 4B  are flowcharts of a test operation order according to an embodiment of the present invention. 
         [0036]    Referring to  FIGS. 4A and 4B , an embedded software test device  100  of the present invention includes an embedded SW analyzing engine  11 , a test case generating engine  12 , a test executing engine  13 , and a test result analyzing engine  14 , all coupled in series.  FIGS. 4A and 4B  further illustrate processes that are performed by each of the foregoing functional blocks. 
         [0037]    The embedded software analyzing engine  11  receives the embedded software  41  from the OS Server  40  in an executable image such as an executable and linking format (ELF). The embedded SW analyzing engine  11  automatically identifies interface locations and debugging information in the embedded software  41  in operation  111 . The embedded SW analyzing engine  11  extracts monitoring symbol information from the executable image in operation  112 . The monitoring symbol information is determined based on a function-call relationship that can be defined according to interface properties. 
         [0038]    The embedded SW analyzing engine  11  extracts caller and callee information associated with each function in the embedded software  40  in operation  113 . 
         [0039]    In operation  114 , the embedded SW analyzing engine  11  identifies functions in each embedded software layer. For example, the embedded software layer is automatically identified as a SUk function in an OS kernel layer in the case of OS system calls and OS API functions. A function that includes a device driver name is automatically identified as a SUd of the device driver layer. Moreover, a function that includes a processor name is automatically identified as a SUp of the HAL layer. 
         [0040]    The embedded SW analyzing engine  11  then extracts an interface type according to a function-call relationship between the embedded software layers in operation  115 . Operation  115  may be based on the interface type definitions illustrated in  FIG. 3 . For example, a function-call relationship between a SUd and a HUa 1  that is related to RAM, or a function-call relationship between SUp and HUa 1  that is related to RAM is automatically extracted as an HPI 1  interface type. On the other hand, a function-call relationship between SUp and HUa 1  that is related to a special register is automatically extracted as an HPI 2  interface type. 
         [0041]    The embedded SW analyzing engine  11  identifies the interface in operation  116 , and finally determines an interface in operation  117 . 
         [0042]    Referring to  FIG. 4A , the test case generating engine  12  automatically generates a test case at each identified interface based on the extracted monitoring symbol information in operation  121 . Then, in operation  122 , the test case generating engine  12  generates a test case that includes pass/fail criteria for the function associated with the monitoring symbol. Each pass/fail criteria may include a set of input data and expected output data. For example, parameters and return values of a function corresponding to an embedded software interface may be limited to registers R 0 , R 1 , R 2 , and R 3  according to standard procedure call protocol that is disclosed in a hardware processor specificaion (and thus may be automatically determined). 
         [0043]    In operation  123 , the test case generating engine  12  converts the test case into a test script for automatically executing the test case in the emulator  20 . The test script includes a script command determining pass/fail of the test result and a script command storing the test result as a log for analyzing the test result. The test script may further include a script commands for stopping at predetermined interface locations in the embedded code, a script command for executing a test target automatically, and a script command monitoring actual output values for comparison to expected output values associated with a pass/fail standard. 
         [0044]    The test case generating engine  12  automatically generates test suite in operation  124 . A test suite is a set of test cases according to an interface coverage. Each test case in the test suite is written in script language for automatically executing in an emulator such that the emulator  20 . Equipped with debugging and monitoring functions. Each test case includes a symbol that will be monitored in a corresponding interface, together with input data and expected output data. 
         [0045]    The process for testing embedded software is continued in  FIG. 4B . The test executing engine  13  associates a test suite with a graphical user interface (GUI) feature, such as a button on a test suite tool bar; upon activation of the GUI feature, the test executing engine  13  executes the selected test suite for a target embedded software by using the emulator  20  in operation  131 . Other GUI features could alternatively be used in operation  131 , according to design choice. 
         [0046]    In operation  132 , the test executing engine  13  stores the test results as a test log. 
         [0047]    In operation  141 , the test result analyzing engine  14  distinguishes a tested interface from an untested interface based on pass/fail information in the test log. The total interface test coverage can then be determined in operation  144  by comparing the number of tested interfaces to the total number of interfaces that were identified in the embedded software. Embedded software interface tests may be continued until 100% of the identified interfaces have been tested. 
         [0048]    The test result analyzing engine  14  analyzes fault locations and causes from the pass/fail information of the test log in operation  142 , and summarizes and/or stores such fault information in operation  145 . 
         [0049]    In operation  143 , the test result analyzing engine  14  calculates performance metrics such as maximum, minimum, and average values based on measured performance data in the test log. The test analyzing engine  14  compiles, stores, and/or reports the calculated performance metrics in operation  146 . 
         [0050]    The process illustrated in  FIGS. 4A and 4B , and described above, may be implemented in processor-executable code (software). The processor-executable code may be stored in a processor-readable storage medium, such a random access memory (RAM), read-only memory (ROM), a hard drive, or a compact disc (CD). 
         [0051]    The invention provides many benefits. The invention provides a method for testing embedded software in actual hardware, so that target hardware can be emulated before finishing a product. The invention also provides a software test automation device for the above automation method. Advantageously, the invention automatically identifies and tests interfaces between different layers of the embedded software. Moreover, the inventive method and system integrates debugging and monitoring functions of an emulator into a test automation tool so that when faults are detected, a fault cause can be suggested. 
         [0052]    The above-disclosed subject matter is to be considered illustrative, and not restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other embodiments, which fall within the true spirit and scope of the present invention. Thus, to the maximum extent allowed by law, the scope of the present invention is to be determined by the broadest permissible interpretation of the following claims and their equivalents, and shall not be restricted or limited by the foregoing detailed description.