Patent Publication Number: US-11640344-B2

Title: Automated testing for content receivers

Description:
BACKGROUND 
     Technical Field 
     The present disclosure generally relates to testing multimedia receiver systems and, in particular, to generating and running test scripts for such systems. 
     Description of the Related Art 
     Providers of television entertainment content typically install a content receiver at a customer&#39;s location, such as a home or business. The content receiver, sometimes called a set top box, receives multi-media signals via coaxial cable, a satellite antenna, a telephone line, or the Internet, for example. The content receiver includes electronic hardware that processes the multimedia signals for viewing on a display such as a television screen, projection device, or monitor. The content receiver also includes proprietary software that controls the hardware and provides an interactive format that a viewer can use, via a remote control, to select programming to watch in real time, or to store for viewing at a later time. Hardware and software inside the content receiver is frequently updated with new versions and new capabilities. For example, some content receivers are more complex than others in that they are equipped to manage the output of multimedia content to be displayed on multiple display devices in multiple locations. There may also be many different content receiver choices based on different service packages available to the consumer. Remote control hardware and software is also frequently updated, and must be compatible with the content receivers. Engineers and manufacturers who produce content receivers therefore need to perform functional testing, evaluation, and troubleshooting of content receivers on a frequent basis, and on a large volume of devices being provided to customers. Performing such tests in a serial fashion, one content receiver at a time, is a time-consuming, labor-intensive, and therefore costly process. 
     BRIEF SUMMARY 
     An automated test platform is disclosed for use by content providers, such as cable and satellite TV companies, for developing and troubleshooting customized software for multimedia content receivers. The automated test platform may be used internally, by a development organization, for example, to perform quality assurance (QA) tests following release of a new version of software. In the testing environment, multiple content receivers, for example, may be arranged in a client-server relationship. The automated test platform allows developers to script test cases that permit interaction with multiple content receivers at the same time. Through the creation of user-configured test scripts, the automated test platform is thus customized to the particular content provider. The automated test platform is used to create scripts, run the scripts on multiple content receivers, and view test results to assess functionality of the content receiver software and/or hardware. A graphical user interface (GUI) is provided that allows technicians without any programming experience to build and run complex interactive test sequences in a modular fashion. Such an automated test platform can be used to test cable and satellite television set top boxes, as well as DVD players, streaming media receivers, and game consoles. 
    
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
       In the drawings, identical reference numbers identify similar elements. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale. 
         FIG.  1    is a schematic illustration of an exemplary residential media entertainment system, according to one embodiment described herein. 
         FIG.  2    is a block diagram showing components of an automated testing system for testing a plurality of content receivers, according to one embodiment described herein. 
         FIG.  3    is a screen shot of a user interface associated with the automated testing system of  FIG.  2   . 
         FIG.  4    is a process flow diagram showing a sequence of steps in a computer-implemented interactive scripting method, according to one embodiment described herein. 
         FIG.  5    is a process flow diagram showing a sequence of steps in a computer-implemented method of testing a plurality of content receivers, according to one embodiment described herein. 
     
    
    
     DETAILED DESCRIPTION 
     In this specification, embodiments of the present disclosure illustrate a subscriber satellite television service as an example. This detailed description is not meant to limit the disclosure to any specific embodiment. The present disclosure is equally applicable to cable television systems, broadcast television systems, Internet streaming media systems, or other television or video distribution systems that include user hardware, typically in the form of a content receiver or set top box that is supported by the media provider or by a third party maintenance service provider. Such hardware can also include, for example, digital video recorder (DVR) devices and/or digital-video-disc (DVD) recording devices or other accessory devices inside, or separate from, the content receiver. 
     Throughout the specification, the terms “subscriber” and “customer” refer to an end user who has a service contract with a media service provider and who has an account associated with the media service provider. Subscriber equipment typically resides at the subscriber&#39;s address. The terms “user” and “viewer” refer to anyone using part or all of the home entertainment system components described herein. 
     One skilled in the art will recognize that the term “signal” refers to any digital or analog signal. Such signals can include, but are not limited to, a bit, a specified set of bits, an A/C signal, or a D/C signal. Uses of the term “signal” in the description can include any of these different interpretations. It will also be understood to one skilled in the art that the terms “connected” and “coupled” are not limited to a physical connection but can refer to any means of communicatively or operatively coupling two devices, for example, a wireless connection. 
     As a general matter, the terms “content receiver,” “television converter,” “receiver,” “set top box,” “television receiving device,” “television receiver,” “television recording device,” “satellite set top box,” “satellite receiver,” “cable set top box,” and “cable receiver” are used interchangeably to refer to a converter device or electronic equipment that has the capacity to acquire, process and distribute one or more television signals transmitted by broadcast, cable, telephone or satellite distributors, or Internet service providers who are a source of streaming media. The terms “digital video recorder (DVR)” and “personal video recorder (PVR)” refer interchangeably to devices that can record and play back television signals and that can implement playback functions including, but not limited to, play, fast-forward, rewind, and pause. As set forth in this specification and the figures pertaining thereto, DVR and PVR functionality or devices can be combined with a content receiver. The signals transmitted by these broadcast, cable, telephone, satellite, or other distributors can include, individually or in any combination, Internet, radio, television or telephonic data, and streaming media. One skilled in the art will recognize that a content receiver can be implemented, for example, as an external self-enclosed unit, a plurality of external self-enclosed units, or as an internal unit housed within a television, a computer, or a mobile computing device. One skilled in the art will further recognize that the present disclosure can apply to analog or digital satellite content receivers. 
     As yet another general matter, it will be understood by one skilled in the art that the term “television” refers to a television set or video display that can contain an integrated television converter device, for example, an internal cable-ready television tuner housed inside a television or, alternatively, that is connected to an external television converter device such as an external set top box connected via cabling to a television. A further example of an external television converter device is the EchoStar Hopper combination satellite set top box and DVR. 
     Turning now to the drawings,  FIG.  1    illustrates one embodiment of a home entertainment setup, which includes a satellite-based multimedia receiver system  200 . The satellite-based multimedia receiver system  200  includes one or more content receivers  202  (two shown,  202   a ,  202   b ), a media presentation device  204  having a display  206 , and a remote control  208 . The satellite-based multimedia receiver system  200  can be located, for example, at a residence, office, or similar location for presenting entertainment media to various users. 
     The content receivers  202 , e.g., a television set top box (STB), can be coupled to the media presentation device  204 , e.g., TV, or the content receivers  202  can be in the form of hardware built into the media presentation device  204 . Each content receiver  202  is communicatively coupled to one or more sources of broadcast media content  210  to receive media content for presentation via the media presentation device  204 . The content receiver can be a satellite STB, a cable STB, a streaming media player, a digital video disk (DVD) player, a digital video recorder (DVR), a game console, or a machine that emulates any one of such devices. Instructions that carry out features of the satellite-based multimedia receiver system  200  are stored in, and/or executed by, components of the content receiver  202 . 
     Sources of broadcast media content  210  can include one or more of a terrestrial television or radio antenna, a satellite television or radio antenna  210   a , a broadband cable subscriber service, streaming media received from the Internet  210   b  directly or via a wireless Internet router, and the like. Media content is provided as a media signal, via communication links  214 , e.g., a satellite antenna communication link  214   a , or an Internet communication link  214   b . The communication links  214  can accommodate a wired signal or a wireless signal. Furthermore, a content provider may provide to the viewer multiple media signals, e.g., via satellite and/or via the Internet as shown, via broadband cable and/or Internet, or in any other suitable way. 
     The satellite receiving antenna  210   a  receives media content via a satellite signal  211  from one or more satellite transponders in earth orbit. Each satellite transponder is, for purposes of the satellite-based multimedia receiver system, a source of content that transmits one or more media channels, such as HBO, ESPN, pay-per-view channels, etc., to the satellite receiving antenna  210   a . A satellite television distributor can transmit one or more satellite television signals to one or more satellites. Satellite television distributors can utilize several satellites to relay the satellite television signals to subscribers. Each satellite can have several transponders. Transponders transmit the satellite signal from the satellite to the satellite receiving antenna  210   a.    
     The broadcast media presentation device  204  having the display  206  can be any electronic device that presents media content to a user; for example, a television, a radio, a computer, a mobile computing device such as a laptop, a tablet, a gaming console, a smart phone, or the like, or the display  206  itself. The display  206  can be any kind of video display device such as a cathode ray tube display, a liquid crystal display (LCD), a plasma display, a television, a computer monitor, a rear projection screen, a front projection screen, a heads-up display, or any other electronic display device. The display  206  can be separate from, or integrated into, the media presentation device  204 . The display  206  can include audio speakers, or the display  206  can be coupled to separate audio speakers. The term “for display” as used herein generally includes presentation of an audio component as well as a video component of the media signal. 
     The remote control  208  is configured to communicate with the content receiver  202  via a wireless connection  212 ; for example, an infrared (IR) signal. The remote control  208  can be operated by a user to cause the content receiver  202  to display received content on the media presentation device  204 . The remote control  208  may also be used to display a programming guide on the display  206  and to communicate program selections to the content receiver  202 . The remote control  208  can also be used to send commands to the content receiver  202 , including channel selections, display settings, and the like. The signal  212  can use, for example, infrared or UHF transmitters within the remote control  208 . One example of an embodiment of the remote control  208  is the EchoStar Technologies Corporation 40.0 Remote Control that includes an IR transmitter and an ultra-high frequency (UHF) transmitter. The remote control  208  may be able to send signals to other peripheral devices that form part of the satellite-based multimedia receiver system  200 . The content receiver  202  may also be able to send signals to the remote control  208 , including, but not limited to, signals to configure the remote control  208  to operate other peripheral devices in the satellite-based multimedia receiver system  200 . In some embodiments, the remote control  208  has a set of Light Emitting Diodes (LEDs). Some remote controls can include Liquid Crystal Displays (LCDs) or other display screens. The remote control  208  can include buttons, dials, or other man-machine interfaces. While the remote control  208  can often be the common means for a user to communicate with the content receiver  202 , one skilled in the art will recognize that other means of communicating with the content receiver  202  are available, including, but not limited to attached keyboards, smart phones, front panel buttons or touch screens. 
       FIG.  2    shows a representative testing system  250 , according to one embodiment. The testing system  250  includes hardware such as a number n of multiple content receivers shown as set top boxes (STBs),  202   a - 202   n , collectively  202 ; control devices  208   a - 208   n , collectively  208 ; processor-based testing modules  252   a - 252   n , collectively  252 ; a network  254 ; a customized application program  256  that executes on a user station  258 ; and a satellite receiving antenna  210  that receives a satellite signal  211  from a content distributor. A satellite TV set top box under test can be coupled to one or more live satellite inputs having different satellite orientations. Embodiments of the testing system  250  may include additional, or fewer, nodes or components than illustrated in  FIG.  2   . 
     The testing system  250  permits a content provider to initiate and control performance of a sequence of functionality tests on the content receivers  202 . Functional testing of the content receivers  202  may address either hardware, software, or both. Functional testing of content receivers may be carried out on new content receiver software emulating an actual content receiver. Alternatively, functional testing of content receivers may be carried out on actual new content receivers manufactured and/or distributed by the content provider, prior to sale. Alternatively, functional testing of content receivers may be carried out on used content receivers that are identified as malfunctioning, for example, units that are returned by customers, or units that have been returned by previous customers and are being prepared for re-issue to new customers. Alternatively, functional testing of content receivers may be carried out remotely on content receivers that are currently in use and identified by customers as malfunctioning. Remote testing can be accomplished via a communications network, for example. Accordingly, the testing system  250  allows for testing of content receiver features before and after installation at a customer location. 
     The content receivers  202  can be accessed or controlled using the control devices  208 , which may be embodied as remote control devices, smart phones, mobile computing devices, personal computers (PCs), or custom peripheral control devices, for example. Each control device  208  is coupled to a respective content receiver  202  and is programmed to deliver commands to that content receiver  202 . The processor-based testing modules  252  are basic processor-based nodes coupled to the network  254 , such as, for example, Windows™-based PCs, a Linux™-based Raspberry Pi™ credit-card sized single board computer, or any other basic computing device. The processor-based testing modules  252  need not be equipped with a keyboard, monitor, or other peripheral devices. The processor-based testing modules  252  are therefore also referred to as CPUs. 
     The network  254  can be a packet-switched communications network such as the Internet, an intranet, a wide-area network (WAN), a local-area network (LAN), or other type of data network. 
     The user station  258  can be any type of workstation configured to execute a customized application that tests functionality of the content receivers  202 . The user station  258  may include a display and an input/output device, e.g., a keyboard, mouse, joystick, touch screen, or the like, via which the testing modules  252  receive input and interactive instructions from a user. Alternatively, the user station  258  may be a mobile device such as a tablet computer, or the like. The customized application program  256  implements a method of testing the content receivers  202  that prompts the user to assemble test scripts, initiate running the test scripts, and evaluate test results presented via the display. Assembly of test scripts employs a GUI-based scripting application. The customized application program  256  can be a web-based application, for example, that accesses the processor-based testing modules  252  via the World Wide Web portion of the Internet. Coupling via such a network  254  allows the user station  258  to be located remotely from the system under test. 
       FIG.  3    shows an exemplary screen shot  300  associated with an automated test platform, according to one embodiment. The screen shot  300  shows what a user sees while working at the user station  258  to create a test matrix  302 , run test scripts, and view results. In particular, the exemplary screen shot  300  shows a screen associated with the user station  258  during operation of the GUI-based scripting application. 
     The test matrix  302  is a data structure that stores test command sequences. The test matrix  302  can be made accessible for editing via, for example, a tab  304  located on a window  303 . The test matrix  302 , once it is interactively created, then serves as a program to be carried out by one or more of the processor-based testing modules  252 , e.g., CPUs, via the control devices  208  to perform tests on one or more of the content receivers  202 . The test matrix  302  consists of lists arranged in columns. Each column of the test matrix  302  contains a sequence of steps, or test script,  305  (two shown,  305   a ,  305   b ), for a particular content receiver  202  under test. Each test script  305  includes a sequence of commands that the processor-based testing module  252  will deliver to a corresponding content receiver  202  via the control device  208 . The test script commands, shown herein by way of example, simulate interaction with the content receiver  202  via a remote control, in the same way that a TV viewer would interact with the content receiver. A first exemplary test script  305   a  shown in the center of the screen shot  300  is associated with STB 1 , designated as a server. A second test script,  305   b , is associated with STB 2 , designated as a client. The test script  305   a  is executed starting at the top row of the test matrix  302 , and ending at the bottom row. After the test matrix  302  is created, it can be saved in the memory  308  so that the test can be retrieved for execution at a later time. 
     On the left side of the screen shot  300  is a GUI-based scripting language menu  306  from which a user, e.g., a technician, can select test building blocks  308  from which to construct the test scripts  305   a,b . For example, the technician can employ a mouse, touch pad, stylus, or the like to select, drag, and drop test building blocks  308  from the scripting language menu  306  to the test matrix  302 . Each selection of a test building block  308  then becomes a row in the test matrix  302 . Examples of test building blocks  308  that can be added to the test matrix  302  include commands, input blocks, output blocks, functions, and so on. Commands are carried out as specific interactions with an STB. In addition, the scripting language menu  306  includes conditional blocks  310  that can be added to perform logical functions as part of the test scripts. Such conditional blocks  310  include, for example, “While,” and “If” blocks, e.g.,  308   b , as well as loops, e.g., the “For loop” shown in the scripting language menu  306 , all of which are typical constructs used in computer programming. After a test building block  308  is added from the scripting language menu  306  to the test matrix  302 , the technician may be prompted to enter additional information about how the command described in the test building block  308  is to be carried out, as explained in more detail below. 
     With reference to the example test matrix  302  shown in  FIG.  3   , the technician has created a test sequence  305   a  to test a content receiver STB 1 . The test sequence  305   a  begins with the command “Switch Feed”  308   a . The “Switch Feed” command  308   a  refers to which satellite antenna  210  will be coupled to STB 1  to feed content to STB 1  during the test. After dragging and dropping the command “Switch Feed”  308   a  onto the test script  305   a  STB 1 , the testing system prompted the technician to enter a desired switch position for the source antenna, by a prompt, “Feed switch: _”. The technician has entered a “1” in  308   a  to indicate switching the feed switch to position 1 to engage a first satellite antenna, for example, satellite antenna  210   a . There may be available one satellite antenna or multiple satellite antennas. Different satellite antennas  210  may be configured differently, so that an STB can be tested while receiving signals from various types of satellite antennas. The Feed switch command can then be used to select which antenna configuration to connect to for the present test, or to switch between the different satellite antennas  210 . 
     Next, the technician has added a conditional if-then block  310   a  which will execute a delay step  308   b  if a condition  308   c  is true, or an action command  308   d  if the condition  308   c  is false. The condition  308   c  depends on the result of a “Screen Read” action on STB 2 . The Screen Read action sends a direct remote control command to STB 2  to capture a screen shot of the TV screen for analysis. In particular, the exemplary Screen Read action  308   c  is “Read standby text,” meaning that the screen is checked to determine whether the TV has responded to the remote control command. STB 2  then stands by to wait for a response to appear on the screen. In this way, actual STB behavior can be compared to the behavior that the programmers expect. It is noted that there exists a library of Screen Read commands, each command reading a specific user interface (UI) element of the screen such as, for example, picture, text, and the like. 
     If the condition  308   c  is True, the delay step  308   b  will cause a delay of 100 milliseconds (ms) to occur. It is noted that the delay step  308   b  extends across both columns  305   a  and  305   b , meaning that the delay step  308   b  is executed with respect to both content receivers STB 1  and STB 2  simultaneously. If the condition  308   c  is False, the action command  308   d  will be executed on STB 2 . The action has been entered by the technician as setting a key called “Power,” meaning that STB 2  will be powered on to wake up from a standby mode. 
     Next in the sequence is another Function command,  308   e . The name of the function command  308   e  is “Enter Point dish from Live AV” which orients the satellite antenna  210   a , connected to STB 1 , in a particular direction, followed by a 1000 ms delay,  308   f . The time needed for the satellite antenna  210   a  to change positions can automatically cause a delay to be inserted after every “point dish” function so that the Function step and the Delay step in the sequence may be linked as a unit. Adding the delay inserts a pause in the execution of the test script  305   a  that allows time for STB 1  to perform the requested function. The delays help to prevent a system crash from occurring in response to a sequence of rapid commands being received. 
     The next action unit  308   h  executes a “Browse” function followed by a 500 ms delay, and so on. It is noted that the action “Key: Menu”  308   j  is a shortcut that simulates selecting a main menu screen using a remote control. 
     One of the functions  308  available in the scripting language menu  306 , but not used in the exemplary test scripts  305   a ,  305   b , is a “Front Panel” command that simulates interaction with a set top box via buttons on the front panel. 
     Once a series of modular test script elements is created, a similar GUI can be used to build a more complex test sequence by combining the test script elements in different ways. Each test script element can be represented graphically by an icon, and different tests can then be sequenced by arranging the icons in a desired sequence. 
       FIG.  4    shows a flow chart for a general computer-implemented interactive scripting method  400 , according to one embodiment. The computer-implemented interactive scripting method  400  assists a user in creating test scripts to use in automated testing of a plurality of content receivers at the same time. The computer-implemented interactive scripting method  400  provides a user interface in which a user can easily generate the test matrix  302  in a modular fashion without programming skills. The computer-implemented method  400  is codified in the customized application program  256 , and is executed by the testing system  250 . 
     At  402 , the customized application program  256  receives input from a technician at the user station  258  specifying a number of content receivers  202  to be tested. 
     At  404 , the customized application program  256  generates a test matrix having a matrix dimension based on the number of content receivers specified by the technician. The test matrix  302  has a number of columns equal to the number of content receivers  202  to be tested. 
     At  406 , the scripting language menu  306  is provided to the technician for selection of test building blocks  308  to become rows in the test matrix  302 . 
     At  408  the customized application program  256  receives a sequence of user selections from the scripting language menu  306  for each one of the content receivers  202 . The user selections constitute a sequence of actions to be executed as the test scripts  305 . Each selection occupies a row in the test matrix  302 . 
     At  410 , the test scripts  305  are stored together as a test matrix  305  in a memory. The memory can be located in the user station  258 , or in another memory location such as, for example, cloud storage accessible by the network  254 . 
     At  412 , if additional tests are desired the method  400  is repeated to create new or additional test scripts, which are then also stored for future use, as set forth below and shown in  FIG.  5   . The tests that were created in the sequence shown and described in  FIG.  4    are one source of the tests that will be carried out in the steps of  FIG.  5   ; however, they can be received from other sources as well. 
       FIG.  5    shows a flow chart for a computer-implemented test method  500 , according to one embodiment. The test method  500  tests a plurality of content receivers at the same time by executing a test matrix such as the exemplary test matrix  302  shown in  FIG.  3    and that was created as shown in  FIG.  4   . The computer-implemented test method  500  is codified in the customized application program  256 , and is executed by the testing system  250 . 
     At  502 , a client hardware configuration is identified. 
     At  504 , client resources are analyzed to determine a particular node setup that should be used. For example, if the number of content receivers to be tested is three, the test case may be referred to as a three-node setup, or a 3-STB test case. Because the number of nodes is variable, the test system is a scalable system. 
     At  506 , a testing scheme is determined for carrying out parallel testing of multiple nodes. For example, a first test matrix  302  stored in memory that was initially created for a 2-node setup can be adapted to accommodate a 4-node setup and can then be used to test all four of the nodes at once. Furthermore, a second test matrix  302  designed for a 2-node setup can be designated to test two of the four nodes. 
     At  508 , the test matrix  302  is executed in response to a user command entered at the user station  258 . The tests are executed by the processor-based testing modules  252  which, in turn, employ the control devices  208  to change the state of the STBs and/or the satellite antenna(s)  210  in accordance with the sequence of commands in the test script  305 . In the embodiment shown in  FIG.  3   , the test script  305  is executed in row-order, starting at the top of the matrix, and progressing downward to the bottom of the matrix. 
     At  510 , test results are displayed at the user station  258  for evaluation by the technician. If additional node setups are to be tested using available tests, the method  500  can be repeated. 
     One skilled in the art will recognize that the present disclosure can also apply to other types of systems that offer automated testing capabilities. Such systems can include computer systems, networking systems, telephone systems, industrial equipment systems, medical equipment systems, and the like. While the disclosure shows and describes media-related embodiments, it will be understood by those skilled in the art that various other changes in the form and details may be made therein without departing from the spirit and scope of the invention. 
     The various embodiments described above can be combined to provide further embodiments. All of the U.S. patents, U.S. patent application publications, U.S. patent applications, foreign patents, foreign patent applications and non-patent publications referred to in this specification and/or listed in the Application Data Sheet are incorporated herein by reference, in their entirety. Aspects of the embodiments can be modified, if necessary to employ concepts of the various patents, applications and publications to provide yet further embodiments. 
     It will be appreciated that, although specific embodiments of the present disclosure are described herein for purposes of illustration, various modifications may be made without departing from the spirit and scope of the present disclosure. Accordingly, the present disclosure is not limited except as by the appended claims. 
     These and other changes can be made to the embodiments in light of the above-detailed description. In general, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and the claims, but should be construed to include all possible embodiments along with the full scope of equivalents to which such claims are entitled. Accordingly, the claims are not limited by the disclosure.