Patent Publication Number: US-2011064319-A1

Title: Electronic apparatus, image display method, and content reproduction program

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2009-213620, filed Sep. 15, 2009; the entire contents of which are incorporated herein by reference. 
     FIELD 
     Embodiments described herein relate generally to an image display technique applied to an electronic apparatus, for example, a personal computer. 
     BACKGROUND 
     In recent years, there have been a rapid increase in the number of pixels and a rapid size reduction for image pickup devices such as CCDs (Charge coupled devices) and CMOSs (Complementary metal-oxide semiconductor) image sensors. Thus, high-resolution images can now be taken even using a cellular phone or a notebook personal computer. Moving and still images and the like taken with an image pickup device, that is, what is called digital images, can be stored in a card-type storage medium or hard disk drive (HDD) for compact management. 
     Furthermore, image reproduction apparatuses called digital photoframes or the like have started to prevail. The digital photoframe includes a slot in which a card-type storage medium is accommodated. The digital photoframe functions to sequentially display, at predetermined time intervals, a plurality of still images stored in the storage medium accommodated in the slot. The digital photoframe is utilized as a desktop accessory or the like. For example, personal computers and digital cameras also commonly function to sequentially display a plurality of still images at predetermined intervals in the same manner as that in which the digital photoframe displays images. 
     Jpn. Pat. Appln. KOKAI Publication No. 2008-153920 discloses a “moving image list display apparatus configured to create and display a unique summary of moving images in a list with the correlations among the scenes of the moving images taken into account so that a user can easily compare the moving images with one another” (see Paragraph [0009]). Various mechanisms have been proposed which include the one in Jpn. Pat. Appln. KOKAI Publication No. 2008-153920 and which are configured to select a desired one of a large number of digital data. Thus, application of any of these mechanisms allows efficient selection of those of a large number of digital images managed by HDD which are to be displayed in the same manner as that in which the digital photoframe displays images. 
     The conventional digital photoframes (including the digital photoframe functions of personal computers and digital cameras) display a plurality of digital images selected from a large number of digital images and stored in a card-type storage medium, in accordance with a rigid and simple rule, that is, in order of image pickup date and time, the storage position on the storage medium, or the file name of the digital image. That is, no highly-intelligent image display method has existed which, for example, arranges display target digital images in an appropriate efficient order so that the user can appreciate the digital images more joyfully. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       A general architecture that implements the various feature of the embodiments will now be described with reference to the drawings. The drawings and the associated descriptions are provided to illustrate the embodiments and not to limit the scope of the invention. 
         FIG. 1  is an exemplary diagram showing the appearance of an electronic apparatus according to an embodiment. 
         FIG. 2  is an exemplary diagram showing the system configuration of the electronic apparatus according to the embodiment. 
         FIG. 3  is an exemplary block diagram showing the functional configuration of a content reproduction application program operating on the electronic apparatus according to the present embodiment. 
         FIG. 4  is an exemplary diagram showing an example of configuration of index information used by the content reproduction application program operating on the electronic apparatus according to the present embodiment. 
         FIG. 5  is an exemplary diagram showing an example of a person specification screen for slideshow display shown by the content reproduction application program operating on the electronic apparatus according to the present embodiment. 
         FIG. 6  is an exemplary diagram showing an example of a detailed setting screen for slideshow display shown by the content reproduction application program operating on the electronic apparatus according to the present embodiment. 
         FIG. 7  is an exemplary conceptual drawing illustrating the basic principle of image selection and display performed by the content reproduction application program operating on the electronic apparatus according to the present embodiment. 
         FIG. 8  is an exemplary flowchart showing the procedure of a slideshow display process executed by the content reproduction application program operating on the electronic apparatus according to the present embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     Various embodiments will be described hereinafter with reference to the accompanying drawings. 
     In general, according to one embodiment, an electronic apparatus includes a group creation module, a display image selection module, and a display control module. The group creation module is configured to create groups by classifying a plurality of still images. The display image selection module is configured to select at least one still image to be displayed from the plurality of still images based on a specified selection condition. The display control module is configured to control display of the at least one still image selected by the display image selection module. The display control module arranges the selected at least one still image in units of the groups created by the group creation module in accordance with a first rule, and arranges the selected at least one still image in each group in accordance with a second rule. 
       FIG. 1  is an exemplary diagram showing the appearance of an electronic apparatus according to the embodiment. The electronic apparatus is implemented as a notebook type personal computer  10 , for example. 
     As shown in the computer  10  includes a computer main body  11  and a display unit  12 . The display unit  12  incorporates a display apparatus including a liquid crystal display (LCD)  17 . The display unit  12  is attached to the computer main body  11  so as to be pivotally movable between an open position where the top surface of the computer main body  11  is exposed and a closed position where the top surface of the computer main body  11  is covered. 
     The computer main body  11  has a thin box-like housing with a keyboard  13 , a power button  14 , an input operation panel  15 , a touchpad  16 , loudspeakers  18 A and  18 B, and the like arranged on the top surface of the housing: the power button  14  is used to power on and off the computer  10 . Various operation buttons are provided on the input operation panel  15 . 
     Furthermore, the computer main body  11  includes a universal serial bus (USB) connector  19  provided on the right side surface of the main body  11  and to which a USB cable or a USB device complying with, for example, the USB 2.0 standard is connected. Moreover, the computer main body  11  includes an external display connection terminal (not shown in the drawings) provided on the rear surface of the main body  11  and complying with, for example, the high-definition multimedia interface (HDMI) standard. The external display terminal is used to output digital video signals to an external display. 
       FIG. 2  is an exemplary diagram showing the system configuration of the computer  10 . 
     As shown in  FIG. 2 , the computer  10  includes central processing unit (CPU)  101 , a north bridge  102 , a main memory  103 , a south bridge  104 , graphics processing unit (GPU)  105 , video-random access memory (VRAM)  105 A, a sound controller  106 , basic input/output system-read only memory (BIOS-ROM)  107 , a local area network (LAN) controller  108 , HDD  109 , optical disc drive (ODD)  110 , a USB controller  111 , a wireless LAN controller  112 , an embedded controller/keyboard controller (EC/KBC)  113 , and electrically erasable programmable-ROM (EEPROM)  114 . 
     CPU  101  is a processor configured to control the operation of the computer  10  and to execute an operating system (OS)  201  and various application programs such as a content reproduction application program  202 ; the OS  201  and the application programs are loaded from HDD  109  into the main memory  103 . The content reproduction application program  202  is software providing a function to reproduce, for example, audio/video (AV) contents stored in digital versatile disc (DVD) set in ODD  110 . The content reproduction program  202  also functions to display digital images stored in HDD  109 , in the same manner as that in which what is called a digital photoframe displays images. Furthermore, CPU  101  executes BIOS stored in BIOS-ROM  107 . BIOS is a program for controlling hardware. 
     The north bridge  102  is a bridge device configured to connect a local bus for CPU  101  to the south bridge  104 . The north bridge  102  also contains a memory controller configured to control accesses to the main memory  103 . The north bridge  102  also provides a function to communicate with GPU  105  via a serial bus complying with the PCI EXPRESS standard. GPU  105  is a display controller configured to control LCD  17  used as a display monitor for the computer  10 . Display signals generated by GPU  105  are transmitted to LCD  17 . Furthermore, GPU  105  can transmit digital video signals to an external display apparatus  1  via an HDMI circuit  3  and an HDMI terminal  2 . 
     The HDMI terminal  2  is the above-described external display connection terminal. The HDMI terminal  2  can transmit uncompressed digital video signals and digital audio signals to an external display apparatus  1  such as a television set, via one cable. The HDMI control circuit  3  is an interface configured to transmit digital video signals to the external display apparatus  1  called an HDMI monitor, via the HDMI terminal  2 . 
     The south bridge  104  controls each of the devices on a peripheral component interconnect (PCI) bus and each of the devices on a low pin count (LPC) bus. The south bridge  104  also contains an integrated drive electronics (IDE) controller configured to control HDD  109  and ODD  110 . The south bridge  104  further provides a function to communicate with the sound controller  106 . 
     The sound controller  106  is a sound source device configured to output audio data to be reproduced, to the loudspeakers  18 A and  18 B or the HDMI control circuit  3 . 
     The LAN controller  108  is a wired communication device configured to perform wired communication complying with, for example, the IEEE 802.3 standard. On the other hand, the wireless LAN controller  112  performs wireless communication complying with the IEEE 802.11 g standard. Furthermore, the USB controller  113  communicates with an external apparatus complying with, for example, the USB 2.0 standard (and connected to the USB controller  113  via the USB connector  19 ). For example, the USB controller  113  performs communication and the like required to load digital images managed by a digital camera, that is, the external apparatus, in HDD  109 . 
     The EC/KBC  113  is a single-chip microcomputer in which an embedded controller for power management and a keyboard controller configured to control the keyboard  13  and the touchpad  16  are integrated. The EC/KBC  114  functions to power on and off the computer  10  in response to the user&#39;s operation of the power button  14 . 
     Now, a functional configuration of the content reproduction application program  202  operating on the computer  10  configured as described above will be described with reference to  FIG. 3 . Here, description will be given of one of the functions of the above-described content reproduction program  202 , the function of displaying digital images (still image data  401 ) stored in the HDD  109  in the same manner as that in which what is called a digital photoframe displays images. 
     As shown in  FIG. 3 , the content reproduction application program  202  includes an index processing module  301  and a slideshow control module  302 . 
     The index processing module  301  executes various indexing processes for creating index information used to search the still image data  401  stored in the HDD  109  for a desired digital image. For example, the index processing module  301  extracts a face image from the still image data  401  showing a person. The face image is extracted by, for example, a face detection process of detecting the face image in the still image data  401  and a clipping process of clipping the detected face area from the still image data  401 . The face area can be detected by, for example, analyzing the characteristics of the still image data  401  and searching for an area with characteristics similar to those of a prepared face image characteristic sample. The face image characteristic sample is characteristic data obtained by statistically processing the characteristics of face images of a large number of persons. The still image data  401  shown in  FIG. 3  may be the images of the respective plurality of frames included in moving image data. 
     The results of the indexing process executed by the index processing module  301  are stored in the database  109 A as index information  402 . The database  109 A is a storage area prepared in the HDD  109  and configured to store the index information  402 .  FIG. 4  shows an example of the configuration of the index information  402  stored in the database  109 A. 
     An “Image ID” is identification information uniquely assigned to each of the still images  401 , and an “Image pickup date and time” is time information indicative of the image pickup date and time of each still image  401 . If the still image data  401  are the images of the respective plurality of frames included in the moving image data, for example, time stamp information indicative of elapsed time from a leading frame is stored in the “Image ID” field+the “Image pickup date and time” field. 
     If the still image data  401  shows persons, “Face image information” is recorded for all of the persons shown in the still image data  401 . As the “Face image information”, first, the face images extracted from the still image data  401  by the index processing module  301  are stored. Furthermore, the index processing module  301  outputs the front level (indicative of the level at which the image of the face is taken from the front) and size (the size of the face image area in the original still image) of each face image extracted from the still image data  401 . Moreover, the index processing module  301  divides the plurality of face images extracted from the still image data  401 , into classes, that is, groups each of images estimated to show the same person. The index processing module  301  then outputs the results of the classification as classification information. The front level, the size, and the classification information are further stored as the “Face image information”. 
     Furthermore, the index processing module  301  determines whether or not each of the still image data  401  contains any character, for example, a character on a sign, and outputs the result of the determination. The result of the determination is stored as “Character information”. A character can be detected by searching for an area with characteristics similar to those of a characteristic sample of each character. 
     Moreover, based on, for example, the image pickup date and time, the index processing module  301  divides the plurality of still image data  401  stored in the HDD  109  into groups. The index processing module  301  then outputs information required to identify the groups. The information is stored as “Group information”. For example, if two temporally consecutive still image data  401  last a period exceeding a predetermined value, the data are divided into two groups. Furthermore, if the still image data  401  are the images of the respective plurality of frames included in the moving image data, then for example, what is called a scene change portion is detected in which the characteristics of the image change significantly between two consecutive images. Then, each scene is determined to be an interval for grouping. 
     That is, the index information  402  enables determination of for example, whether or not each still image data  401  contains a face image, that is, whether or not the still image data  401  shows a person, who the person shown in the image is, whether or not the image data  401  contains any character, and which group the image data belongs. In another aspect, the index information  402  can be used to quickly search the plurality of still image data  401  stored in the HDD  109  for, for example, still image data  401  showing the target person, still image data  401  showing no person, or still image data showing the target person and a character. 
     The slideshow control module  302  uses the index information  402  created by the index processing module  301  to select the still image data  401  that meet a predetermined selection condition, from the plurality of still image data  401  stored in the HDD  109 . The slideshow control module  302  thus carries out a display process of sequentially display images in the same manner as that in which what is called a digital photoframe displays images. The operational principle of the slideshow control module  302  will be described below in detail. In the present embodiment, the slideshow refers to sequential display of the plurality of still image data  401  at predetermined intervals. The slideshow may include not only the simple sequential display but also processed display such as a transition effect for display switching. 
     The slideshow control module  302  includes a user interface module  3021 . The slideshow control module  302  uses the user interface module  3021  to display a person specification screen for slideshow display shown in  FIG. 5 , on LCD  17 . 
     As shown in  FIG. 5 , a face list display area “a 1 ” is provided on the person specification screen. The slideshow control module  302  uses the index information  402  stored in the database  109 A to place face images of persons shown in at least one of the plurality of still image data  401  stored in the HDD  109 , on the face list display area “a 1 ” as choices. As shown in  FIG. 4 , the index information  402  includes the front level, the size, and the classification information. For each group of face images with the same classification information, the slideshow control module  302  selects one of the face images with a size equal to or larger than a threshold which has the highest front level, for example. The face images to be placed on the face list display area “a 1 ” can be switched by operating the keyboard  13 , the touchpad  16 , or the like. 
     The face list display area “a 1 ” allows the user to specify any person as search key information required to select still image data  401  to be displayed as a slideshow from the plurality of still image data  401  stored in the HDD  109 . It is assumed that the user desires to display the still images  401  for a person A included in persons A to H with their face images arranged on the face list display area “a 1 ”. In this case, the user selects the face image “a 11 ” of the person A located on the face list display area “a 1 ”. 
     Furthermore, a “Slideshow start” button “a 2 ” and a “Detailed setting” button “a 3 ” are provided on the person specification screen. The “Slideshow start” button “a 2 ” is used to specify the start of a slideshow after one of the face images on the face list display area “a 1 ” has been selected. The “Detailed setting” button “a 3 ” is used to set an expansion condition for at least one of a selection rule for the still image  401  for the person in the face image selected on the face list display area “a 1 ” and a display rule for the selected still image  401 . Operating the “Detailed setting” button “a 3 ” allows the slideshow control module  302  to display a detailed setting screen for slideshow display shown in  FIG. 6 , on LCD  17  using the user interface module  3021 . 
     As shown in  FIG. 6 , a scene list display area “b 1 ” is provided on the detailed setting screen. On the scene list display area “b 1 ”, an expansion condition for the selection or display rule suitable for each scene can be set; when “Travel” is selected, many scenery images (showing no person) are selected and are displayed before the other images (regardless of the image pickup order) so as to allow the place shown in the images to be immediately and clearly determined, and when “Party” is selected, images are selected so as to cover all the party participants, and images showing characters or the largest number of persons are displayed before the other images (regardless of the image pickup order) so as to allow the user to immediately and clearly determine what party it is. Here, it is assumed that the “Detailed setting” button “a 3 ” is operated on the person specification screen shown in  FIG. 5  and that “Travel” (icon “b 11 ”) is selected on the detailed setting screen. 
     The detailed setting screen with the scene list display area “b 1 ” as described above includes a “Person specification” button “b 2 ” used to return to the person specification screen shown in  FIG. 5 . Thus, the following procedure can also be carried out on the personal specification screen shown in  FIG. 5 . First, the “Detailed setting” button “a 3 ” is operated to display the detailed setting screen. An expansion condition is set. Then, the “Person specification” button “b 2 ” is operated to return to the person specification screen. A person is then selected. 
     Furthermore, as is the case with the person specification screen shown in  FIG. 5 , a “Slideshow start” button “b 3 ” is provided on the detailed setting screen. Thus, the start of a slideshow can be specified without the need to return to the person specification screen shown in  FIG. 5 . 
     Now, with reference to  FIG. 7 , description will be given of the basic principle of how upon receiving the operation on the personal specification screen shown in  FIG. 5  and the operation on the detailed setting screen shown in  FIG. 6 , the slideshow control module  302  selects the still image data  401  to be displayed and display the selected still image data  401 . 
     In  FIG. 7 , “A” is a first conceptual drawing showing that the plurality of still image data  401  stored in HDD  109  are time-sequentially arranged and grouped in order of image pickup date and time based on the index information  402  stored in the database  109 A. As shown in “A” in  FIG. 7 , it is assumed that the plurality of still image data  401  stored in the HDD  109  are divided into three groups, a group (n), a group (n+1), and a group (n+2). Each of the boxes with numbers 1, 2, 3, . . . in each group expresses one still image data  401 . Each of the numbers denotes the order of the image pickup date and time in the group. 
     Furthermore, the circle in the box of the still image data  401  shown at reference number “c 1 ” indicates that this still image data  401  is an image showing the person specified on the person specification screen shown in  FIG. 5 . The triangle in the box of the still image data  401  shown at reference number “c 2 ” indicates that this still image data  401  is an image not showing the person specified on person specification screen shown in  FIG. 5  but showing any other person. The rhombus in the box of the still image data shown at reference number “c 3 ” indicates that this still image data  401  is an image showing no person. 
     That is, “A” in  FIG. 7  indicates that two of the groups (n), (n+1), and (n+2), that is, the groups (n) and (n+2), show the person specified on the person specification screen shown in  FIG. 5 . Thus, the slideshow control module  302  first carries out narrowing-down to determine the still image data  401  in the groups (n) and (n+2) to be display targets (selection of the display target group). 
     The slideshow control module  302  determines that the still image data  401  belonging to the same group as that which includes the still image data  401  showing the person specified on the person specification screen shown in  FIG. 5  are images having a connection with the specified person (even if the images do not show the specified person). The slideshow control module  302  selects the still image data  401  from the selected display target group based on the index information  402  stored in the data base  109 A, in accordance with an upper limit specified for the number of selected image data. At this time, highly intelligent image search that is different from the simple selection of an image showing the person is performed as follows. In principle, the image data  401  showing the person specified on the person specification screen shown in  FIG. 5  is preferentially specified. If for example, “Travel” is selected on the detailed setting screen shown in  FIG. 6 , many scenery images (showing no person) are selected. 
     In  FIG. 7 , “B” is a second conceptual drawing showing the results of selection of the still image data  401  performed by the slideshow control module  302 . As shown in “B” in  FIG. 7 , in the group (n), an image has been selected which is shown at reference number “d 1 ” and which does not show the person specified on the person specification screen shown in  FIG. 5  but show any other person. Furthermore, an image has also been selected which is shown at reference number “d 2 ” and which shows no person. Similarly, in the group (n+2), an image has been selected which is shown at reference number “d 3 ” and which does not show the person specified on the person specification screen shown in  FIG. 5  but show any other person. Furthermore, an image has also been selected which is shown at reference number “d 4 ” and which shows no person. 
     According to conventional image search techniques, the images shown at reference numbers “d 1 ” to “d 4 ” are not selected if the person is specified as search key information. In this regard, the content reproduction application program  202  carries out highly intelligent, effective image search such that a slideshow is displayed so as to clearly show “to where with whom”. 
     Furthermore, in  FIG. 7 , “C” is a third conceptual drawing showing the order of display of the still image data  401  performed by the slideshow control module  302 . As shown in “C” in  FIG. 7 , in order to meet a demand to view the latest image first, the slideshow control module  302  forms an arrangement in which the groups are placed from the current to the past (group (n+2), group (n)) and an arrangement in which the images in each group are placed so as to allow the user to enjoy viewing the images more naturally. That is, different algorithms are adopted for the arrangement of the groups and for the arrangement of the images in each group. 
     As described above, it is assumed herein that “Travel” is selected on the detailed setting screen shown in  FIG. 6 . Thus, in order to allow the user to determine, for example, “to where” first, the slideshow control module  302  places, for the group (n+2), the image shown at reference number “d 4 ” and showing no person, that is, the image estimated to be a scenery image, at the leading position. The slideshow control module  302  subsequently arranges the other images in order of image pickup date and time. Similarly, for the group (n), the slideshow control module  302  places the image shown at reference number “d 2 ” and estimated to be a scenery image, at the leading position, and subsequently arranges the other images in order of image pickup date and time. 
     That is, the content reproduction application program  202  also performs highly intelligent, effective image display such that an image suitable for understanding a scene such as “travel” is placed at the leading position as a typical image. If the setting of an expansion condition on the detailed setting screen shown in  FIG. 6  is not carried out, the images in each group may be arranged in order of image pickup date and time. 
       FIG. 8  is an exemplary flowchart showing the procedure of a slideshow display process executed by the content reproduction application program  202 . 
     The content reproduction application program  202  first uses the index information  402  stored in the database  109 A in the HDD  109  to display, in a list as choices, the face images of the persons shown in at least one of the plurality of still image data  401  stored in the HDD  109  (block A 1 ). 
     When one of the face images displayed in the list is selected (block A 2 ), the content reproduction application program  202  uses the index information  402  stored in the database  109 A in the HDD  109  to determine the group including the still image data  401  in which the person with the selected face image is shown (block A 3 ). The content reproduction application program  202  then selects the display target still image data  401  from the determined group (block A 4 ). 
     Furthermore, the content reproduction application program  202  uses the index information  402  stored in the database  109 A in the HDD  109  to further determine the order in which the selected still image data  401  are displayed (block A 5 ). The content reproduction application program  202  then sequentially displays the selected still image data  401  on LCD  17  in accordance with the determined display order (block A 6 ). 
     As described above, the computer  10  can realize highly intelligent image search and display such that selection is made not only of the image group that can be detected directly using the specified search key information but also of images with a predetermined connection with the search key information and that the images are arranged so as to allow the user to appreciate the image group more joyfully, instead of in the simple order of, for example, image pickup date and time. 
     In the above-described example, in order to allow display target still image data  401  to be selected from the still image data  401  stored in the HDD  109  using the index information  402  stored in the database  109 A in the HDD  109 , the face images of the persons shown in the still image data  401  are displayed in a list as choices. However, the usage of the index information  402  (used to select the display target still image data  401  from the still image data  401  stored in the HDD  109 ) is not limited to this aspect and may be varied in many ways. 
     For example, a table in which the classification information on the face images is associated with the names of the persons may be stored in the database  109 A as the index information  402 . Then, the names of the persons can be displayed in a list as choices. To manage the table, user interface module may be provided which displays face images in a list so that the name of the person in any of the face images can be input. 
     Furthermore, for example, the following method may be used instead of specifying a face image or name (which specifies a target person) as search key information. A date is specified as search key information, and monthly group information is used as the index information  402 . Then, the selected and displayed image data may include not only the still image data  401  with an image pickup date and time matching the specified date but also the still image data  401  obtained in the same year and month or in a different year but the same month and considered to have a connection with one another. 
     The various modules of the systems described herein can be implemented as software applications, hardware and/or software modules, or components on one or more computers, such as servers. While the various modules are illustrated separately, they may share some or all of the same underlying logic or code. 
     While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel embodiments described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the embodiments described herein may be made without departing from the spirit of the inventions. 
     The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.