Abstract:
A terminal device includes a display unit having at least a first display area and a second display area, a detection unit configured to detect a viewing state in which the first display area and/or the second display area are being viewed, and a control unit configured to, when a predetermined application is executed, perform control on the basis of the viewing state detected by the detection unit, for switching between a state in which display is performed on either one of the first display area and the second display area and display is not performed on the other one of the first display area and the second display area and a state in which display is performed on both the first display area and the second display area. A method and computer program storage device holding computer readable instructions cooperate to form a system that performs these functions.

Description:
CROSS REFERENCE TO RELATED PATENT APPLICATION 
       [0001]    The present application claims the benefit of the earlier filing date of U.S. provisional patent application Ser. No. 61/350,952, filed in the USPTO on Jun. 3, 2010, the entire contents of which being incorporated herein by reference. 
     
    
     BACKGROUND 
       [0002]    1. Field of the Invention 
         [0003]    The present invention relates to a terminal device and a display method to be applied to, for example, a terminal device having a display unit including two display areas. The present invention also relates to an application program to be executed in the terminal device. 
         [0004]    2. Description of the Related Art 
         [0005]    Various terminal devices, such as a mobile phone terminal device, an electronic book terminal device, and a mobile game device, which have two display units have been proposed and are in practical use. 
         [0006]    Two display units allow a terminal device to provide various forms of display and satisfactory display quality. For example, using two display units, an electronic book terminal device can display two pages in a book, which improves operability allowing a user to view the electronic book in the same way as viewing a printed book. 
         [0007]    Japanese Unexamined Patent Application Publication No. 2009-251273 discusses a computer device having a multi-display in which images are displayed on a plurality of displays in a color tone that can save power consumption. 
       SUMMARY 
       [0008]    In fabricating a terminal device having two display units, a large amount of electrical power is necessary to display images on the two display units, and as a result power consumption of the terminal device may increase. In particular, since, in general, such a terminal device runs on a built-in battery, multiple display panels may shorten the working time of the battery. 
         [0009]    The present invention has been made in view of the above circumstances, as recognized by the present inventors. Accordingly, it is desirable to both improve operability given by multiple display units and prevent an increase in power consumption. 
         [0010]    Aspects of the present invention may be applied to a terminal device having a display unit provided with at least a first display area and a second display area and a detection unit configured to detect a viewing state in which the first display area and/or the second display area are being viewed. 
         [0011]    When a predetermined application is executed, control is performed on the basis of the viewing state detected by the detection unit, for switching between a state in which display is performed on either one of the first and second display areas and display is not performed on the other one of the first and second display areas and a state in which display is performed on both the first and second display areas. 
         [0012]    According to an embodiment of the present invention, when the user views only one of the two display areas, display is performed on only the display area viewed by the user and display on the other display area is temporarily discontinued. When the user views both the two display areas to the same degree, display is simultaneously performed on both the display areas. 
         [0013]    According to an embodiment of the present invention, when a user views only one of two display areas, display is performed on the display area viewed by the user. At this time, display is not performed on the other one of the display areas and thus electric power necessary for display can be reduced. When the user views both the two display areas to the same degree, display is performed on both the display areas. This can allow the user to view two displayed images when it is necessary to compare the images and realizes efficient reduction of power consumption without causing the user inconvenience. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0014]      FIG. 1  is a block diagram illustrating an example of a configuration according to an embodiment of the present invention; 
           [0015]      FIG. 2  is a perspective view illustrating an example of an appearance of a terminal device according to an embodiment of the present invention; 
           [0016]      FIG. 3  is a block diagram illustrating a control configuration according to an embodiment of the present invention; 
           [0017]      FIG. 4  is a flowchart illustrating a procedure of control operation according to an embodiment of the present invention; 
           [0018]      FIGS. 5A and 5B  illustrate a first example of a detection state and a display state according to an embodiment of the present invention; 
           [0019]      FIGS. 6A and 6B  illustrate a second example of a detection state and a display state according to an embodiment of the present invention; 
           [0020]      FIGS. 7A and 7B  illustrate a third example of a detection state and a display state according to an embodiment of the present invention; 
           [0021]      FIGS. 8A to 8C  illustrate an example of a change in a display state according to an embodiment of the present invention; 
           [0022]      FIG. 9  illustrates an example of a change in a display state according to a modification of an embodiment of the present invention; and 
           [0023]      FIGS. 10A and 10B  illustrate an example of a change in a display state according to another modification of an embodiment of the present invention. 
       
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENT 
       [0024]    An embodiment of the present invention will be described in the following order. 
         [0025]    1. description of terminal device ( FIG. 1  to  FIG. 3 ) 
         [0026]    2. description of control of display change ( FIG. 4  to  FIG. 8B ) 
         [0027]    3. description of first modification ( FIG. 9 ) 
         [0028]    4. description of second modification ( FIGS. 10A and 103 ) 
         [0029]    5. other modifications 
       [1. Description of Terminal Device] 
       [0030]      FIG. 1  illustrates an example of a configuration of a terminal device  100  according to an embodiment of the present invention.  FIG. 2  illustrates an example of an appearance of the terminal device  100 . 
         [0031]    As illustrated in  FIG. 2 , the terminal device  100  includes a left panel  200  and a right panel  300  composed of two casings which can be folded together. The terminal device  100  also has a camera  102  disposed at the fold between the left panel  200  and the right panel  300 . The camera  102  picks up an image of a user facing the display panels. In the present embodiment, the camera  102  (or other type of sensor such as an image detector) is used to detect a state in which the user is viewing the display panels. 
         [0032]    Referring to  FIG. 1 , the terminal device  100  has the left panel  200  and the right panel  300 . 
         [0033]    The left panel  200  has a left control unit  201 , a touch panel unit  202 , a liquid crystal display (LCD) panel unit  203 , and a backlight unit  204 . An image is displayed on the liquid crystal display panel unit  203  under the control of the left control unit  201 . At this time, control for turning on the backlight unit  204  is also performed. The touch panel unit  202  detects contact by a finger or the like on the surface of the liquid crystal display panel unit  203 . 
         [0034]    The right panel  300  has a right control unit  301 , a touch panel unit  302 , a liquid crystal display panel unit  303 , and a backlight unit  304 . An image is displayed on the liquid crystal display panel unit  303  under the control of the right control unit  301 . At this time, control for turning on the backlight unit  304  is also performed. The touch panel  302  detects contact by a finger or the like on the surface of the liquid crystal display panel unit  303 . 
         [0035]    The terminal device  100  has the camera  102  and a camera control unit  101  which controls imaging operation of the camera  102  and performs processing of an image signal obtained through imaging operation. The camera control unit  101  also performs detection of a gaze point of the user and control of display on the basis the detection according to the present embodiment. The camera control unit  101  performs the detection and control by sending a control command to the left control unit  201  and the right control unit  301 . When the camera  102  is used as a gaze point detection unit, the camera control unit  101  determines an image obtained by the camera  102  and detects a gaze unit. 
         [0036]    The terminal device  100  also has a wireless phone control unit  103 , a wireless communication unit  104 , and an antenna  105  so as to have a wireless phone function. For example, using the wireless communication unit  104 , the terminal device  100  may receive data such as image data and electronic book data to be displayed on the liquid crystal display panel units  203  and  303 . 
         [0037]    Each of the control units illustrated in  FIG. 1  has a memory which stores a program necessary for control, an application program, and the like. 
         [0038]      FIG. 3  is a functional block diagram illustrating a configuration for executing software. 
         [0039]    A face position task  192  receives an instruction from a camera driver  191  and sends an instruction for controlling display to each of the left panel  200  and the right panel  300 . In the present embodiment, the face position task  192  controls backlight by sending an instruction to a left backlight driver  291  and a right backlight driver  391 . At the same time, the face position task  192  sends an instruction to each of a left LCD panel driver  292  and a right LCD panel driver  392  to individually control display on the left and right panels. The face position task  192  similarly controls contact detection using a left touch panel driver  293  and a right touch panel driver  393 . 
       [2. Description of Control of Display Change] 
       [0040]    In the following, an example of a procedure of a display control processing according to the present embodiment will be described with reference to a flowchart in  FIG. 4 . 
         [0041]    The processing procedure illustrated in  FIG. 4  is performed when a predetermined application program is executed. In the processing, the face of a user picked up by the camera  102  (Step S 11 ) is detected and the vertical axis (X axis) of one frame of the picked up image and the center of the face in the picked up image are compared (Step S 12 ). When it is determined that the center of the face of the user deviates from the vertical axis to the right to a predetermined degree or more and that the gaze point is located on the right panel, the processing procedure proceeds to processing of Step S 13  to Step S 15 . 
         [0042]    Specifically, the backlight unit  204  of the left panel  200  is turned off (Step S 13 ), the touch panel unit  202  of the left panel  200  is turned off (Step S 14 ), and the liquid crystal display panel unit  203  of the left panel  200  is turned off (Step S 15 ). After a predetermined standby time period elapses, the processing procedure returns to the detection of Step S 11  and the following processing is repeated. 
         [0043]    If it is determined in the comparison in Step S 12  that the position of the face of the user deviates from the vertical axis to the left to a predetermined degree or more and that the gaze point is located on the left panel, the processing procedure proceeds to processing of Step S 16  to Step S 18 . 
         [0044]    Specifically, the backlight unit  304  of the right panel  300  is turned off (Step S 16 ), the touch panel unit  302  of the right panel  300  is turned off (Step S 17 ), and the liquid crystal display panel unit  303  of the right panel  300  is turned off (Step S 18 ). After a predetermined standby time period elapses, the processing procedure returns to the detection of Step S 11  and the following processing is repeated. 
         [0045]    When the processing in the flowchart in  FIG. 4  is repeated and display on one of the panels (other than the one that has been turned off) is turned off, the other one is turned on, so that display is performed on either one of the panels. The orders of the processing of Step S 13  through Step S 15  and the processing of Step S 16  through Step S 18  illustrated in  FIG. 4  are merely examples and may be altered. 
         [0046]    Although not illustrated in the flowchart in  FIG. 4 , when it is determined in the comparison in Step S 12  that the face of the user is positioned at substantially the center with respect to the vertical axis and that the gaze point is located at around the center, display is simultaneously performed on both the left panel  200  and the right panel  300 . Specifically, the touch panel unit  202 , the liquid crystal display panel unit  203 , and the backlight unit  204  of the left panel  20  are turned on, and at the same time, the touch panel unit  302 , the liquid crystal display panel unit  303 , and the backlight unit  304  of the right panel  300  are turned on. After display on both the panels is performed, the processing procedure returns to the standby processing of Step S 19  and then returns to the face detection of Step S 11 . 
         [0047]      FIG. 5A  to  FIG. 7B  each illustrate an example of a face detection state and a display state. 
         [0048]      FIGS. 5A and 5B  illustrate the case where it is determined that the user&#39;s face in a picked up image is positioned at substantially the center and the gaze point is located at around the center. In this example, a single common display panel is used to display one or both pages of reading material, such as a book. Thus, one display panel that is subdivided into two pages may be used, in addition to two separate panels. One example of a common display panel is a table computer. The process for implementing the detection and backlight management may be implemented in the form of a downloadable application. 
         [0049]    Specifically, as illustrated in  FIG. 5A , when the center of the face is detected at a position around the center between the positions of the two panels, activation screens of the currently activated application are displayed on both the left and right panels as illustrated in  FIG. 5B . 
         [0050]      FIGS. 6A and 6B  illustrate the case where it is determined that the user&#39;s face in a picked up image is deviated to the left and the gaze point is located on the left panel. 
         [0051]    Specifically, as illustrated in  FIG. 6A , when the center of the face is detected at a position that is deviated to the left from the center between the positions of the two panels, an activation screen of the currently executed application is displayed on the left panel and display on the right panel is discontinued as illustrated in  FIG. 6B . 
         [0052]      FIGS. 7A and 7B  illustrate the case where it is determined that the user&#39;s face in a picked up image is deviated to the right and the gaze point is located on the right panel. 
         [0053]    Specifically, as illustrated in  FIG. 7A , when the center of the face is detected at a position that is deviated to the right from the center between the positions of the two panels, an activation screen of the currently executed application is displayed on the right panel and display on the left panel is discontinued, as illustrated in  FIG. 7B . 
         [0054]      FIGS. 8A to 8C  illustrate an example of changes in the display state. For example, the terminal device  100  displays page  1  on the right panel  300  and page  2  on the left panel  200 . At this time, when the user views both the pages, both the pages are displayed as illustrated in  FIG. 8A . 
         [0055]    When the user views only page  1  on the right panel  300 , only page  1  on the right is displayed and display of page  2  on the left is discontinued, as illustrated in  FIG. 8B . Further, when the user views only page  2  on the left, only page  2  on the left is displayed and display of page  1  on the right is discontinued. 
         [0056]    As described above, according to the present embodiment, panels are switched on or off depending on where the user is viewing on two screens, so that the unnecessary panel is turned off. Thus, when the user views the two screens at the same time, the two screens are displayed simultaneously with satisfactory image quality. When the user views only one of the two screens, the corresponding one of the panels is turned on and the other panel is turned off, which reduces power consumption. The simultaneous display on the two panels can immediately be recovered when necessary while one of the panels is off. Therefore, the off-state of one of the panels produces little inconvenience. 
         [0057]    In the present embodiment, such a display manner is adapted only when an application program such as an electronic book which allows either one of display panels to be turned off is executed. Thus, switching on or off of display screens is not performed when an application program for comparing images on two display panels is executed, which also causes the user little inconvenience. 
       [3. Description of First Modification] 
       [0058]    In the following, an example of a modification of an embodiment of the present invention will be described with reference to  FIG. 9 . 
         [0059]    In the example of display changes illustrated in FIGS.  8 A to  8 C, a display panels to be turned off is simply selected. However, in the example illustrated in  FIG. 9 , a displayed image is changed in accordance with switching of panels to be turned off. 
         [0060]    Specifically, as illustrated in drawings in  FIG. 9 , from the top down to the bottom, it is assumed that page  1  is displayed on the right panel in the initial state. Then, when the gaze point is moved to the left, the display state changes to the one illustrated in the second drawing, in which page  2  is displayed on the left panel. This change is the same as has been described using  FIGS. 8A to 8C . Further, when the gaze point is moved to the right, the display state changes to the one illustrated in the third drawing, in which page  3  is displayed on the right panel. Thereafter, in accordance with left-right movement of the gaze point, display of pages proceeds to page  4 , page  5 , and so forth, on a one-page by one-page basis. 
         [0061]    Thus, page forwarding can be done on the basis of only the movement of the gaze point of the user, which improves operability. Moreover, the detected gaze point is used to trigger a “page turn” event, in which the system recognizes that the user has finished looking at one page, and wishes to move to a next successive page as indicated by the user turning his or her head to look at the next panel (either left or right panel, depending on whether the left or right page is being viewed). 
       [4. Description of Second Modification] 
       [0062]    In the following, another modification of an embodiment of the present invention will be described with reference to  FIGS. 10A and 10B . 
         [0063]    In this example, the orientation of the terminal device  100  in  FIG. 2  is changed by 90 degrees, and the left panel  200  is arranged at the top and the right panel  300  is arranged at the bottom. Although the camera  102  is arranged at the center in  FIGS. 10A and 10B , the camera  102  may be disposed at an end position as illustrated in  FIG. 2 . In this example, the ranges of the gaze point of a user to be detected are around the center, at a position higher than the center, and at a position lower than the center. 
         [0064]    Using the above arrangement, an application program in which a question and an answer to the question are displayed is to be executed. The user is supposed to view the liquid crystal display panel unit  203  on the top to see the question. As illustrated in  FIG. 10A , the liquid crystal display panel unit  303  on the bottom is turned off and displays no images while the question is displayed on the liquid crystal display panel unit  203  on the top. 
         [0065]    When the user moves the gaze point to the bottom, display on the liquid crystal display panel unit  303  on the bottom is initiated and the answer to the question is displayed on the liquid crystal display panel unit  303 , as illustrated in  FIG. 108 . At this time, display on the liquid crystal display panel unit  203  is continued so that the question on the top and the answer on the bottom are displayed simultaneously. Alternatively, the liquid crystal display panel unit  203  on the top may be turned off so that the display of the question is discontinued. 
         [0066]    In this way, by switching on or off display of questions and answers in accordance with movement of the gaze point of the user, display processing of the application program for displaying questions and answers can be appropriately performed without manual operation of keys, touch panels and the like by the user. Moreover, power can be saved by turning off either one of the panels. 
         [0067]    Note that the above application program for displaying questions and answers is described as an example, and this modification example may be applied to any application program in which changing display along with study progress is preferred. 
       [5. Other Modifications] 
       [0068]    In the processing described using the flowchart in  FIG. 4 , the display panels are turned off to turn off display of images as well as the backlight. However, it is sufficient to turn off the backlight in order to turn off display of images. That is, since a large amount of power is consumed by the backlight in a liquid crystal display panel, only turning off backlight produces a sufficient effect. 
         [0069]    Moreover, instead of completely turning off display of images, the images may be displayed in a darker state by reducing the brightness of one of the display panels. To reduce the brightness of the panel, the brightness of the backlight may be reduced. Alternatively, when a self-emitting display panel such as an organic electroluminescent image display panel is used, power consumption may be reduced by reducing the luminance of display elements. 
         [0070]    The configuration illustrated in  FIG. 1  is applied to a terminal device serving as a mobile phone terminal device having a wireless communication function. However, the configuration may also be applied to any terminal device having two display units even if it does not have a communication function. 
         [0071]    Moreover, the configuration illustrated in  FIG. 1  and  FIG. 2  is applied to a physically separate display panels. However, the configuration may be applied to a display panel having two divided display areas such as first and second display areas. Then on and off of display are controlled in accordance with whether the gaze point is located on the first display area or the second display area. 
         [0072]    In the embodiment described above, the gaze point is detected on the basis of the position of the face of the user picked up by a camera. However, the gaze point of the user may be detected by other processing. For example, the direction of the line of sight may be detected by an image picked up by a camera and the gaze point may be detected on the basis of the detected direction. 
         [0073]    Furthermore, any available terminal device having a camera or the like capable of detecting a gaze point may be installed with application software which executes processing of the above embodiment so as to similarly function as a dedicated terminal device. When the terminal device has a communication function, the application software may be downloaded to the device using the communication function. When the terminal device does not have a communication function, the application software may be stored in a storage medium such as a memory card and then transferred to the terminal device. 
         [0074]    It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.