Abstract:
A method and system providing enhanced data for visual displays are provided. A remote selector identifies a plurality of objects and sends at least one request to retrieve data about the objects. A server receives the request from the remote selector and retrieves the data about the objects using one or more unique index keys from a database. A display device displays the data provided by the server, and the data concurrently appears with the objects on the display device.

Description:
PRIORITY INFORMATION 
       [0001]    This application claims priority from provisional application Ser. No. 61/567,166 filed Dec. 6, 2011, which is incorporated herein by reference in its entirety. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    The invention is related to the of field visual display systems, and in particular a system and method for providing enhanced data for visual displays. With the rapid advance in science and technology, electronic devices have gained great popularity, and people are becoming more and more dependent on electronic devices such as TV, video recorder/player, and/or multi-media AV device. In order to control electronic devices at a distance from the user, remote control devices are provided for the user to select a corresponding remote control function, such as previous/next channels, adjusting sound volume or switching channels, with the key on the remote control device. For generally known remote control devices, the user remotely controls electronic devices by pressing the keys on the remote control device. Thus, if the user would like to switching channels, adjusting volume or switching to previous/next channels, the user needs to look at the remote control device, select the corresponding of the desired function, and then presses the key to remotely control the electronic device. 
       SUMMARY OF THE INVENTION 
       [0003]    According to one aspect of the invention, there is a method of providing enhanced data for visual displays. The method includes identifying a plurality of objects. The method includes assigning a unique key for each of the objects in a database. The method includes requesting the key from the database. Additionally, the method includes determining data associated with the key. Further, the method includes providing the data to a display device, and the data concurrently appears with the objects on the display device. 
         [0004]    According to another aspect of the invention, there is a system of providing enhanced data for visual displays. The system includes a remote selector that identifies a plurality of objects and sends at least one request to retrieve data about the objects. The system includes a server that receives the request from the remote selector and retrieves the data about the objects using one or more unique index keys from a database. Further, the system includes a display device that displays the data provided by the server, and the data concurrently appears with the objects on the display device. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0005]      FIG. 1  is an illustrated embodiment of an example of a plurality of servers in communication with a database; 
           [0006]      FIG. 2  is an illustrated embodiment of an example of a control module in communication with a database and a cloud; 
           [0007]      FIG. 3  is an illustrated embodiment of an example of a plurality of databases in communication with a central database; 
           [0008]      FIG. 4  is an illustrated embodiment of an example of providing identity and background data about a person; 
           [0009]      FIG. 5  is a schematic illustration of one embodiment of a remote control depicting a plurality of buttons that serve a plurality of functions; 
           [0010]      FIG. 6  is a schematic illustration of one embodiment of a plurality of functions performed by a plurality of buttons of a remote control; 
           [0011]      FIG. 7  is an illustrated embodiment of an example of a process to add or insert a record to a database; 
           [0012]      FIG. 8  is an illustrated embodiment of an example of a process to delete a record from a database, and an example of a process to edit a record in a database; and 
           [0013]      FIG. 9  is an illustrated embodiment of an example of a central database in communication with a plurality of databases and a user input terminal. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0014]    It is to be understood that the terminology used herein is for the purpose of describing particular embodiments of the invention only, and is not intended to be limiting. 
         [0015]    As will be appreciated by one having ordinary skill in the art, the present invention is directed to a system and method for providing enhanced data for visual displays. By the use of a visual marker that may be activated by a viewer, a pre-programmed database with pre-programmed associations may be invoked to display textual and other data relevant to a person, place or object. 
         [0016]    The system uses a preprogrammed database index that associates people, places and objects to the database using a unique index key for each object. The database may make the association of the people, places and objects on TV screens, computer screens and any digital media or display devices. Examples of other implementations include but are not limited to: television shows for sports, movies, TV series, and talk shows, as well as computer displayed screens such as internet displays, including touch screens. If an object is selected on the display device but no unique key has been assigned by the index then a number of steps will be followed to enable services to that object: 
         [0017]    1) Best guess for a match based on visual recognition of the object-for instance the blender on the counter in the movie mat be a GE blender, but since GE does not participate in the service, the ndex and software can suggest they look up a Hamilton Beach blender—this provides the user or viewer with the closet object identification and has the added benefit of providing Hamilton Beach an advertising opportunity even when their product is not in the visual display. 
         [0018]    2) Alternate keys approach will try to combine non-exact matches to make a best guess for the object, but has the risk of identifying a hammer as a screwdriver. 
         [0019]    Referring to  FIG. 1 , a computer or television screen  2  is depicted, where the screen  2  with a movie, TV show or sports show is playing. The screen  2  can be part of a television system having a cable or TWO box  4  or the screen can be part of a computer system that is internet-enabled where there is no cable or TIVO box  4 . After connecting to an internet website  6 , the latest information about all products for sale is provided, without having a user to enter such data themselves. It is organized by the manufacturer and mapped to UPC or SKU codes, through a database  8  that is indexed by unique keys for each object and/or person. 
         [0020]    For example, one implementation of the invention is a user viewing an online copy of the most recent Best Buy circular (Best Buy may create their circular) and connecting it online to a tagging system or the like for latest information about all the products offered for sale, without having to enter the data themselves. As a result, this turns the Best Buy circular into an online, interactive ad instead of a flat, static online publication. The database  8  is in communication with partner servers  10 , such as IMDB or ESPN. Further, the database  8  is in communication with a smart ad server  12 , which may display full or mini-commercials or links to web pages. 
         [0021]    Referring to  FIG. 2 , all interfaces are through a cloud or the internet  14 . A control module  16  manages all operations between the database  8  and internet/network using a number of interfaces. As will be described in further detail below, the database  8  may be a standard database that is either local or remote; the schema provided in Table 1 is what makes the database  8  unique. The scheme along with decisions rules that can be learned, pre-programmed or edited to create new ones or improve database access rules makes this service novel. Also, the database  8  can be in a cloud based service that can be replicated, backed up and preserved so a future service can be offered to show what the state of the database  8  was some time in the past. A later version of the database  8  can be created by appending or each unique database record key a time stamp, so each time it gets updated, changed or deleted, there is a timestamp—thus allowing the cloud based service to provide a lookup that is current. 
         [0022]    Referring to  FIG. 3 , the database  8 , which is where input/output operations are performed, is in communication with the ad server  12  and a controller module  20 . In addition to the function described above, the ad server  12  is also used to store advertisements that may be used to replace or supplement traditional advertising. The ad server  12  provides a link to a viewing mechanism  18 , such as a digital TV, DVD, computer, iPad or tablet. The database  8  is in communication with a controller module  20 , which is responsible for at least application program interface (API) processing that ensures the unique key, adding a unique key, eliminating dupes, and adding or updating fields. A plurality of bulk uploads are fed into the controller module  20 , such as from Amazon  22 , Overstock  24 , IMDB  26  and Netflix  28 . Additionally, information from a user or operator  30  is fed to controller  20 . 
       People-Focused Implementations 
       [0023]    Referring now to  FIG. 4 , there is shown an embodiment of a representation of identity and background data  32  about actors on a television show, along with a focus marker  34 . As illustrated in  FIG. 5 , there is shown an embodiment of a representation of a standard remote control  36  that a television viewer may use, where the existing interactive keys on the remote control  36  display identity and background data  32  about the actors on, for example, a television show. 
         [0024]    More specifically, those existing interactive A, B and C buttons ( 38 ,  40 , and  42 , respectively) on the remote control  36  may be used to move from object to object or object to person. For example, button A  38  may move a focus marker  34  among people and objects, and button B  40  may launch a data service. In an alternate setup, button A  38  may invoke a focus marker  34 , the arrow buttons  44  on the remote control  36  may be used to choose between people and objects on the screen, and button B  40  may launch a data service. Button C  42  may be used to choose among displayed informational topics, as may the arrow buttons  44  on the remote control  36 . In addition, a filter may be added which may allow the viewer to determine whether they would like to view people only, objects only, or both. 
         [0025]    By accessing existing API https://dev.twitter.com/, one can use the tagged twitter handle of an actor to access his or her twitter stream. The invention allows for merging the twitter handle with a live action movie, or live TV show that includes that person is truly unique. For instance, Brad Pitt may be walking down the Red Carpet at the Academy awards and through the invention one can overlay that live action TV show with his twitter stream by connecting the API to https://twitter.com/bradpittupdates twitter handle and stream those updates. This is a rather elegant merge of the feed from twitter and combined with the novel live matching service described henceforth of the Brad Pitt actor key by looking up in a database the twitter handle—opening a connection to the twitter feed and then displaying it live as an overlay (on top of) the TV show. 
         [0026]    Other implementations include sports programs, movies, TV shows, and live news programs to the extent that voice-to-text translation technology may be implemented and associated. For instance, a newscaster may say “Libya—Moammar Gadhafi” and the invention software may associate the text translation of Moammar Gadhafi=P — 44129870, which is an index key to the database  8  at the cloud server  14 . 
       Object-Focused Implementations 
       [0027]    Using the remote control  36  to launch display of additional data, the TV viewer may launch a pop up display of data, such as the stores an appliance (e.g. a mixer) may be bought at or websites where it may be purchased, along with prices. 
         [0028]    Another example is a viewer watching a TV cooking show where a cake is being baked. The cake may be selected to be displayed in a frame(s) of the show—an option that may pop up is the recipe. Further, since baking recipes may require special handling in high altitude, such a recipe may be included to provide bake times tailored to the specific location of the viewer. 
         [0029]    Accordingly, this invention may replace traditional advertising. There are many advantages to this new form of advertising placement. First, TV shows may replace lost revenue that is suffered because of time shifting. Secondly, commercials may be updated in real time, because the ads do not reside at the broadcaster—they are presented by an ad server  12  which may provide ad data in text, sound or video formats or a link to another website—all in near real time. 
         [0030]    The ads may be served region-specific or even person-specific; for instance, once it is known that the viewer is clicking on a, for example KitchenAid mixer, that is being viewed in the TV show, and an IP address of the viewer is also known, then the product and location of the vendor may be provided. This information may generate results such as price in an area of the viewer, stores in an area of the viewer, or even models specific to an area of the viewer. 
         [0031]    If the viewer were viewing a TV show that, for example showed a car chase using a late model or currently available vehicles, for example a BMW  750  chasing a Lexus, a viewer may be interested in further information about the Lexus. The viewer may “select” the Lexus, along with another selection of the location of the viewer (derived by an IP address), where it is possible to deduce that the viewer is in, for example California, and display vehicle information, gas mileage and emission information. The cost of gasoline for a year may be computed by knowing the location of the viewer and the mileage rating of the car. 
         [0032]    Enhanced, replacement or derived advertising is a major benefit of this invention, and may possibly generate revenues for broadcasters using the invention. 
         [0033]    One example is a TV show, not live, such as a TV series. Display in any scene or frame may include actors, places and objects. For example, a TV show may have, a kitchen scene, and in the sceneappears a KitchenAid mixer. Prior to the show being displayed, during post-production, using provided existing video editing software, the object will be assigned an index key associated with the database  8  using the controller  20 . 
       Use of Unique Database Keys 
       [0034]    The unique keys are a critical and innovative component of the invention. 
         [0035]    The keys may be held in a cloud server  14  for the shared but unique licensees of this invention. The benefits of a centrally assigned unique key are basic: no duplicates accidently assigned, no multiple keys assigned to the same UPC or SKU, or vehicle model, and the shared benefit of licensees of not having to enter the same information over and over again. Table 1 below illustrates a database schema layout, with the associated descriptions and field names for the data. Note that the invention can include a database schema and timestamp arrangement as well for older databases structures. 
         [0000]    
       
         
               
               
               
               
             
               
               
               
               
             
           
               
                 TABLE 1 
               
               
                   
               
               
                 Data 
                 Description 
                 Column # 
                 Field name 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                 K_090890987A 
                 Server key that is unique and the 
                 1 
                 Primary_key 
               
               
                   
                 index to other pointers below than 
               
               
                   
                 can be used to point or access other 
               
               
                   
                 database 
               
               
                 Mixed or UPC 
                 Product_id unique 
                 2 
                 UPC_SKU 
               
               
                 symbol or SKU 
               
               
                 Momar kaddifi 
                 People - not unique 
                 3 
                 Name 
               
               
                 Sy19546 
                 Twitter handle - unique on twitter 
                 4 
                 Twitter_id 
               
               
                 Superman001 
                 Facebook password - not unique 
                 5 
                 FB_psw 
               
               
                 Momar-kadaffi-libya 
                 IMDB Actor index key - not unique 
                 6 
                 Actor_name 
               
               
                 031-25-1235 
                 Social security number - unique 
                 7 
                 SSN 
               
               
                   
                 Amazon product key, used to order 
                 8 
                 Alt_key_amazon 
               
               
                   
                 an object or movie off of 
               
               
                   
                 Amazon.com. For instance, if this 
               
               
                   
                 were someone clicking on a 
               
               
                   
                 kitchen aid mixer that they saw on 
               
               
                   
                 a cooking show, they then could 
               
               
                   
                 order it off Amazon without doing 
               
               
                   
                 an Amazon search 
               
               
                 98323983982938293 
                 ICEC_number - this is a number 
                 9 
                 ICEC 
               
               
                   
                 assigned to phones and other 
               
               
                   
                 electronic equipment, it can be 
               
               
                   
                 utilized for custom purposes - 
               
               
                   
                 usually with a phone 
               
               
                   
                 Alternate keys that can be used for 
                 10 
                 Alt_key_2 
               
               
                   
                 lookup, may or may not be unique 
               
               
                   
                 and may produce multiple results 
               
               
                   
                 Alternate keys that can be used for 
                 11 
                 Alt_key_3 
               
               
                   
                 lookup, may or may not be unique 
               
               
                   
                 and may produce multiple results 
               
               
                   
                 Price_discounted_price_sale_price; 
                 12 
                 Price_data 
               
               
                   
                 this field will hold pricing data for 
               
               
                   
                 objects, for example a mixer 
               
               
                 Boston Ma 
                 Location or store location; used to 
                 13 
                 Location_data 
               
               
                   
                 show where the object was seen, 
               
               
                   
                 such as Boston or the store location 
               
               
                   
                 it may be bought in 
               
               
                 Ad for Levi&#39;s jeans 
                 Advertisement_link, this field can 
                 14 
                 Ad_link 
               
               
                 as worn in TV show 
                 connect the product or object to a 
               
               
                 being watched live 
                 video or digital ad, this is a key 
               
               
                 or on DVD or 
                 aspect of the invention as it can 
               
               
                 downloaded from 
                 replace traditional advertising in 
               
               
                 iTunes 
                 TV shows, and also carry forward 
               
               
                   
                 the option for a person to see 
               
               
                   
                 advertisements even when the 
               
               
                   
                 show is on DVD or ITunes format. 
               
               
                   
               
             
          
         
       
     
         [0036]    As an example of a use of this technology, the maker of for example KitchenAid mixers, may request a unique key from the cloud server  14  for its new model of mixer. Then, resellers of that product, such as Sears, KMART, Amazon and Target, may obtain this information from a database server  8 , managed and stored in the cloud server  14 , which may be accessed in real time or ahead of time. 
       Tagging System and Software 
       [0037]    The centralized database server that is used in post processing of movies, TV shows and the like utilizes a specialized, unique key for each object—assigned using the software. The video editing software may tag the data stream with people, product, key words or script words. In the above example the database is the same large cloud based database that is a major part of the invention, with the defined schema above. Moreover, the term software is used as an example of using existing facial and object recognition software that is enhanced to automatically assign keys and details in the database to video images or digital version of shows or streams and have all objects automatically tagged. When objects are not recognized, human intervention using the same “software” can be required. The existing facial or object recognition software is not part of the invention, modifying it to recognize and then assign database keys or associate with the existing database entries is part of the invention. 
         [0038]    Another implementation may extend the above capability to magazines online; however, that is not core to this invention. 
         [0039]    Referring to  FIG. 6 , the viewer may press button A  38  from the remote control  36  and the focus marker  34  will appear above the person in the screen. 
         [0040]    Then, the viewer may press button B  40  from the remote control  36 , and the details  46  that pop up will show up super-imposed over an image  48  of, for example Jason Bourne, next to the actor, or in a corner of the screen. 
         [0041]    In another embodiment, the method of control may be through an external device, such as a keyboard. The keyboard may be connected to a computer or a Smart or Internet enabled TV, which may be done via a wireless keyboard that is connected to the TV or computer using Bluetooth or another wireless protocol. 
         [0042]    In another embodiment, an implementation of the keyboard may be a virtual keyboard that is displayed on the TV or computer screen and accessed using the remote control  36 . 
         [0043]    Other implementations of the keyboard function can include using virtual keyboards through a tablet like an iPad, or an Android tablet device, or using an iPhone with the existing interfaces that replicate a full or modified keyboard, iTouch, or other smart phone or smart phone devices that may create and enable a virtual or screen based keyboard. 
       Database Operations 
       [0044]    Database operations are designed to provide a unique database key for each object that is tagged or identified, and to ensure that the same object does not get two keys assigned to it. In the cases where an exact match is not made a series of tests can be done by the software: 1) Best guess for a match based on visual recognition of the object-for instance the blender on the counter in the movie maybe a GE blender, but since GE does not participate in the service, our index and software will suggest they look up a Hamilton Beach blender—this provides the user or viewer with the closet object identification and has the added benefit of providing Hamilton Beach an advertising opportunity even when their product is not in the visual display; and 2) Alternate keys approach will try to combine non-exact matches to make a best guess for the object, but has the risk of identifying a hammer as a screwdriver. 
         [0045]    The software tool that runs on a computer, with IOS, Linux, or Windows, that recognizes objects in a video stream or, using a mouse, allows the user or viewer to mark the relative area of the object with mouse clicks. This is not dissimilar to video or photo editing software that allows the user of the software to edit images or objects, for digital enhancement. For the purposes of the software and invention, it adds a unique key column 1 of the database layout above in Table 1—then it may add additional data fields in at least two methods: 
         [0046]    Method 1: A human sitting at the terminal may manually enter every field. 
         [0047]    Method 2: An operator of the software may select, from a drop down list, a related database  8  that contains all the other details. Options and examples include: 
         [0000]    SKU database, UPC database, Amazon database, Store database (such as Macy&#39;s), Overstock.com database. 
         [0048]    This may be accomplished by a set of APIs and data tranfers done allowing for the storing of information between databases  8  and  22 - 30 . This attribute of the invention is a data conversion and entry method that is proprietary to the invention and the database. Frequent updates can be made to these published data sources using common API&#39;s or custom one&#39;s built to extract the data and normalize it into the cloud database of this invention. 
         [0049]    The database  8  works like a standard, high performance, replicated database in many ways, but the unique ways are as follows: 
         [0050]    1. Allow database attributes to be added manually by operator of software mentioned above. 
         [0051]    2. Allow singular updates to records from other database sources using custom API&#39;s that may be used to seamlessly add many fields about one object, but only for purposes of required operations. For example, the API may add several fields from the IMDB.com database about an actor, but would not at that time include the detailed data about the actor, such as movies, co-stars, date of birth, DOD, birthplace—all that data would be linked over on a per request basis under this invention from IMDB.com. 
         [0052]    3. Enable bulk updates of new records from bulk sources. For instance, each day Amazon.com adds many new items, movies or books to its database, the API will enable the updates, additions, deletes and edits in a bulk mannner from Amazon.com. 
         [0053]    4. Allow editors to edit incorrect entries to the database. 
         [0054]    5. Database integrity operations would constantly check all updates and edits to ensure that primary database keys are unique. The updates and edits may be done one at a time by a user or in massive volume by an external provider. For instance, bulk updates may be accepted from Amazon.com and NetFlix on the same day, and both may be trying to add a new movies release, for example “The Greatest Minds of IP”. Although it is perfectly acceptable for both Amazon and NetFlix to send this update, the APIs will ensure that only one unique entry is made with two alternate keys—one for Amazon, one for NetFlix. 
       Process Steps for Database Operations 
       [0055]    Referring to  FIG. 7 , the following process steps are used for adding or inserting  50  a record:
   a. Check field attributes {name, description, price, dob, SKU or UPC). SKU or UPC will suffice most of the time.   b. If the added attempt finds that it is a dupe, it will not add the record—it will be written to a report file for error reporting that may be reviewed by data providers or the human user.   c. If the data record is new and correct, it will be added. The new key is from a store of keys that is held in another file on the database.   
 
         [0059]    Referring to  FIG. 8 , the following process steps are used for deleting  52  a record:
   a. Delete solely by a match on unique key {See Table 1}.   b. If there is a certain match the record will be deleted from the database by removing the pointers, but not the physical database record; clean-up may occur later.   
 
         [0062]    Referring to  FIG. 8 , the following process steps are used for editing a record:
   a. Verify record match by unique key, or search by alternate key {see Table 1}.   b. Database software uses field name match to update that field with new data.   
 
         [0065]    Referring to  FIG. 9 , the following process steps are used for creating ad server data:
   a. Used for operations that provide advertising for the object.   b. The ad server data {see Table 1} provides a description and hyperlink to location on the internet of the advertisment. The Amazon database  56 , the SKU/UPC database  58 , and the IMDB database  60  all communicate with the internet  14 , which a main database  62  also communicates with. This database  62 , which communicates with a user input terminal  64 , has records of the database layout schema provided in Table 1.   
 
         [0068]    All software programs and system applications described herein are platform independent and can execute in any browser such as Firefox®, Internet Explorer®, or Chrome®. The web applicant can also be written in any platform independent-base computer language, such as Java or the like. All software programs and system applications execute on a client computer using a processor or the like. All software programs and system applications can be stored in the RAM or ROM of the client. Furthermore, all software programs and system applications can be stored on an external memory device to be uploaded to the client computer for execution. The elements associated with all software programs and system applications can be executed from the client computer or on a remote server. All database servers can be local on the client computer or on a remote server, as well as on a system remote from either the client computer or remote server. The client computer, remote server, and database servers communicate to each other using known communication protocols, such as TCP/IP. 
         [0069]    Although the present invention has been shown and described with respect to several preferred embodiments thereof, various changes, omissions and additions to the form and detail thereof, may be made therein, without departing from the spirit and scope of the invention.