Abstract:
A method of taking an image of a scene that includes an item of interest, includes providing a digital imaging device having an imaging lens and a viewing screen and electronically providing an alignment feature on the viewing screen. The alignment feature is configured to allow a viewer to see both the alignment feature and an image of the item of interest on the viewing screen. The method includes adjusting at least one parameter from the group consisting of angular position of the digital imaging device with respect to the item of interest and angular orientation of the digital imaging device to achieve alignment of the image of the item of interest on the viewing screen with the alignment feature wherein the alignment feature has a structure so that the adjusting to achieve alignment restricts to exclusively one angular position of the digital imaging device with respect to the item of interest and exclusively one angular orientation of the digital imaging device. The method also includes capturing the image of the item of interest.

Description:
FIELD 
       [0001]    This patent application generally relates to a programmable computer system for providing a set of images. More particularly, it relates to a system that creates a consistent set of images for members of a type of product. Even more particularly, it relates to a system that creates a natural sounding human voice recording describing products along with the corresponding consistent set of images. 
       BACKGROUND 
       [0002]    The world wide web has provided the possibility of providing useful written, audio, and visual information about a product that is offered for sale, such as real estate, as described in “Automatic Audio Content Creation and Delivery System,” PCT/AU2006/000547, Publication Number WO 2006/116796, to Steven Mitchell, et al, published 9 Nov.,  2006  (“the &#39;547 PCT application”). The &#39;547 PCT application describes an information system that takes in information from clients and uses this information to automatically create a useful written description and matching spoken audible electronic signal, and in certain cases a matching visual graphical display, relating to the subject matter to be communicated to users. The information system transmits this information to users using various communications channels, including but not limited to the public telephone system, the internet and various retail (“in-store” or “shop window” based) audio-visual display units. A particular aspect of the &#39;547 PCT application relates to an automated information system that creates useful written descriptions and spoken audio electronic signals relating to real estate assets being offered for sale or lease. 
         [0003]    US Patent Application 2008/019845, “System and Method for Generating Advertisements for Use in Broadcast Media” to Charles M. Hengel et al, filed 3 May 2007 (“the &#39;845 application), describes systems and methods for generating advertisements for use in broadcast media. The method comprises receiving an advertisement script at an online system; receiving a selection indicating a voice characteristic; and converting the advertisement script to an audio track using the selected voice characteristic. 
         [0004]    Commonly assigned “Automatic Creation of Audio Files, U.S. Pat. No. 8,112,279, (“the &#39;279 patent”), incorporated herein by reference, that will issue on Feb. 7, 2012, provides ways to automatically generate audio files and to combine them with images. 
         [0005]    Applicants recognized that better schemes are needed to create video that can be automatically combined with the audio descriptions of products of the &#39;279 patent, and such solutions are provided by the following description. 
       SUMMARY 
       [0006]    One aspect of the present patent application is a method of taking an image of a scene that includes an item of interest. The method includes providing a digital imaging device having an imaging lens and a viewing screen, and electronically providing an alignment feature on the viewing screen. The alignment feature is configured to allow a viewer to see both the alignment feature and an image of the item of interest on the viewing screen The method includes adjusting at least one parameter from the group consisting of angular position of the digital imaging device with respect to the item of interest and angular orientation of the digital imaging device to achieve alignment of the image of the item of interest on the viewing screen with the alignment feature wherein the alignment feature has a structure so that the adjusting to achieve alignment restricts to exclusively one angular position of the digital imaging device with respect to the item of interest and exclusively one angular orientation of the digital imaging device. The method also includes capturing the image of the item of interest. 
         [0007]    Another aspect is a method of building a description of a particular product of a class of products on a website, wherein the website is on a server. The method includes reading a code on the particular product and transmitting data derived from the reading. The method includes determining information about the particular product from the transmitted data. The method also includes selecting a plurality of alignment features based on the information. The method also includes providing a digital imaging device, wherein the digital imaging device includes a viewing screen. The method also includes providing an alignment feature on the viewing screen, adjusting location of the digital imaging device, and capturing an image of the particular product when the image of the particular product is aligned with the alignment feature and repeating the capturing an image of the particular product for the plurality of alignment features to obtain a plurality of images of the particular product. The method also includes transmitting the plurality of images of the particular product to a web site. 
         [0008]    Another aspect is a computer-usable medium having computer readable instructions stored thereon for execution by a processor to perform a method of building an audio-visual description of a particular product corresponding to one or both of the above methods. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0009]    The foregoing will be apparent from the following detailed description, as illustrated in the accompanying drawings, in which: 
           [0010]      FIG. 1   a  is a flow chart illustrating one embodiment of a system that provides a uniform set of images of different products of the same type and combines that set of images with an automatically generated audio to create an audio-visual product description of a vehicle; 
           [0011]      FIG. 1   b  provides more detail to the flow chart of  FIG. 1   a  for the portion that is used to create a set of images of the product; 
           [0012]      FIGS. 2   a ,  3   a  are views of a screen of a digital image device showing examples of ghost semi-transparent overlay structures; and 
           [0013]      FIGS. 2   b ,  3   b  are views of a screen of a digital image device showing examples of images of an actual vehicle positioned so they are overlapped by the ghost semi-transparent overlay structures of  FIGS. 2   a ,  3   a.    
       
    
    
     DETAILED DESCRIPTION 
       [0014]    The present applicants created a system that provides a set of images for each product of a group of products, such as vehicles of a particular type on a dealer lot, so they all have substantially the same photographic views of the product. The images so obtained can then be provided to populate a web site so that a customer can compare different makes and models of that type of vehicle with consistent sequencing and photographic views. 
         [0015]    In one embodiment, different types of vehicles, such as sedans and trucks, may have different sequencing and/or photographic views to illustrate features that may or may not be present for another type of vehicles. Within a particular type of vehicle the content and number of photographic views captured may depend on the presence or absence of certain attributes on the specific vehicle. For example certain models of a vehicle may have a specialized interior for which additional images may be captured than for models of that vehicle without that specialized interior. 
         [0016]    A fairly complete set of attributes of a vehicle is available from information that resides electronically in a data base that can be accessed based on the particular vehicle&#39;s Vehicle Identification Number (VIN). The VIN may be used in embodiments of the present patent application to help determine which of several sets of photographic views are to be used. 
         [0017]    In one embodiment, substantially the same views for all the vehicles of a particular type are obtained by providing a set of alignment features, such as ghost semi-transparent overlay structures on the screen of a digital imaging device to which the actual image of the vehicle on the screen is aligned before the image is captured. Alternatively, alignment features can include overlay outlines. In either case, an alignment feature for a vehicle includes an outline corresponding to the shape of the vehicle from a predetermined location or angle to which an actual image of the vehicle can be aligned, as further described herein below. They may include any number of views, including exterior views, interior views, an odometer view, an engine view, and underside views. The exterior views may include three-dimensional views from predetermined angles. 
         [0018]    A variety of types of digital imaging devices, such as a digital camera, a smart phone, and a tablet computer can be used. In one embodiment, the digital imaging device can communicate with the internet or be connected for such communication. 
         [0019]    In one embodiment, the order and number of images of the vehicle captured corresponds to the order and number of alignment features in the set of alignment features provided. The alignment features may themselves be derived from images of the vehicle captured from different angles or isolating different parts of the vehicle. 
         [0020]    In one embodiment, a voice description of some of all of the images in the set is obtained by automatically providing a sequence of stitched together audio fragments that describe the particular features, or attribute values, of the specific vehicle as each of those attributes is viewed in an imageof the set of images, as described in commonly assigned U.S. Pat. No. 8,112,279 (“the &#39;279 patent”), incorporated herein by reference. In one embodiment described in the &#39;279 patent, the voice description automatic creation scheme obtains the attribute values of each specific vehicle from information that resides electronically in the data base that can be accessed based on the particular vehicle&#39;s VIN. 
         [0021]    In one embodiment the set of images obtained by the method of the present patent application is combined with audio obtained with the automatic audio generation scheme of the &#39;279 patent to provide a detailed audio-visual description of the vehicle. While the present application illustrates the ideas of the applicants with reference to vehicles, such as automobiles and trucks, the ideas can be applied to other types of vehicles, such as buses, snow vehicles, boats and other water craft, airplanes and other types of aircraft, trains, and earth moving equipment. The ideas can also be applied to other types of products, such as industrial equipment, appliances, electronic devices, farm animals, and real estate. 
         [0022]    In this patent application, the term “attribute” refers to a feature of a product that can be one of several choices. 
         [0023]    The term “attribute value” refers to the specific one of the different choices of an attribute. 
         [0024]    The term “voiceover product description” refers to a human voice audio description of a specific product. 
         [0025]    The term “fragment” refers to one or more words intended to be spoken in order as part of a voiceover product description. 
         [0026]    The term “audio fragment” refers to an audio file containing a fragment that was recorded by a human. 
         [0027]    The term “stitch” as used in this patent application refers to the process of concatenating audio fragments, for example, to produce the voiceover product description. For stitching two or more audio fragments together the audio fragments and their order are specified and their contents stored in a single output file that includes all of the content from the audio fragments, non-overlapping, and in the specified order. The term stitch is also used referring to the similar process of concatenating video files. 
         [0028]    The term “automatic” refers to a process executed by a computer with no human intervention. 
         [0029]    The phrase “capture an image” refers to taking a picture and recording the image in a memory. 
         [0030]    In one embodiment, a uniform set of product videos of all the new or used vehicles of a particular type on a dealer&#39;s lot can be obtained from the set of images of each vehicle obtained as described in this patent application combined with audio description of that vehicle obtained with the method of the &#39;279 patent. For accomplishing this the attributes of each vehicle for both audio and images are found on an electronically available source based on a product identification code, such as the VIN. For other types of products, such as electronic devices, equipment, appliances, and real estate, the product serial number, product model number, or real estate code number could be used to locate product description information that resides electronically to obtain the attribute values and to determine the set of images to be captured. 
         [0031]    Thus, under one embodiment of the present application built by the present applicants, a person uses a digital imaging device to capture the set of images of a particular vehicle according to the set of alignment features generated for that particular type of vehicle from the VIN. The entire video, with its audio and video tracks, is then automatically generated without further human involvement. Human input to answer questions about the product, to generate a script about the product, or to combine images with audio was avoided. 
         [0032]    For example, using the system of the present patent application, the voiceover product description:
       “This four door sedan features a four speed transmission and front wheel drive. It has a 2.4 liter engine, a sunroof, mag wheels, and a spoiler,”
 
can be built up from audio fragments found in separate audio recordings for each of the attributes while images of the vehicle corresponding to each attribute value mentioned are sequentially displayed as that attribute value is described.
       
 
         [0034]    For a new or a used car, much of the needed information may available electronically from the manufacturer based on VIN. Additional information can be added from online data bases, such as accident history. Other additional information can be provided electronically, such as when a used car is added to a dealer inventory data base, including VIN, mileage, whether the vehicle has any dents or scratches, dealer enhancements, and the set of images captured using the method of this patent application. Information in this dealer inventory data base can also be drawn upon for audio description creation. Thus, the full audio description can include up to date information about the used vehicle, such as, “this car has been driven fewer than 25,000 miles” while an imageof the odometer is shown and “this car has dealer installed rust proof undercoating” while an imageof the underside is shown. 
         [0035]    The setup part of the process described in this patent application is performed by humans, and it provides voice recordings and directions for using the voice recordings that will be used to assemble the voiceover product descriptions for all the various specific products, as described in the &#39;279 patent. The directions include specifying the contents of a common template and specifying rules for inclusion of audio fragments in the voiceover product description. The directions also include providing the images from the set of images captured by the dealer. 
         [0036]    The automated part of the process is performed by a computer running software that can be configured to execute the automated steps for many different vehicles with no human intervention to provide a voiceover product description for each of the specific vehicles and the images that correspond in order and timing with the audio description. 
         [0037]    More than one computer can be used to provide parallel processing and faster creation of the thousands of voiceover product descriptions of the images used to describe thousands of vehicles. 
         [0038]    The present applicants recognized that the number of different car possibilities far exceeds the number of different variable elements for a car. For example, there are about 30 different car manufacturers and 3 different door configurations which gives 90 different car combinations possible for just those two attributes. Yet there are only 33 different individual attribute values. 
         [0039]    An actual car can have about 50 different relevant attributes that might be of interest to a customer, and can be varied by the manufacturer or by the dealer, including year, manufacturer, model, color, body style, doors, transmission, wheel drive, engine type, engine size, number of cylinders, air conditioning, power sun roof, power windows, power windows, mirrors, and door locks, keyless entry, rain sensing wipers, spoiler, roof rack, upholstery, CD player, radio, antitheft devices, stability control, antilock brakes, and warrantee. 
         [0040]    Since many of these attributes can be chosen independently of the others, this means that millions or billions of combinations of these 50 different relevant attributes can be chosen. However, even if there are an average of ten choices for each attribute value, for about 50 attributes there are only about 500 different individual attribute values altogether. Thus, by making only about 500 voice recordings the present applicants recognized that, with appropriate automatic stitching, they could create human voice descriptions of any of the possible car combinations. Based on the information in the data base for a particular VIN, appropriate ones of the 500 voice recordings can be selected and stitched together to automatically provide the description of any particular vehicle that can have any of those millions or billions of car possibilities. The present applicants recognized that they could therefore create a relatively small number of human voice recordings during setup and then, based on information obtained electronically from the VIN, automatically stitch together the appropriate voice recordings to make an accurate audio voiceover product description of any car or truck or for any other type of product. The set of images generated by the embodiments of this patent application can be combined with that voice recording to create the audio-visual product. 
         [0041]    In the automatic steps described below, for a vehicle having a particular VIN, the computer will find the vehicle&#39;s attribute value for each attribute that appears in the common template. For example, the computer will find the actual model year of the particular vehicle, as provided in data residing electronically based on that particular VIN. The computer will apply rules to determine which audio fragments are applicable with that particular vehicle based on its attribute values. When the computer determines the model year of the vehicle with that particular VIN it will not include fragments in the result that indicate other model years. 
         [0042]    The result of this step is a single “way” file with an authentic sounding human voice description of the vehicle. Based on the stitched together files, that voice description might say, “This 2008 Honda Accord has 4 doors and room for 5 passengers. It has less than 10,000 miles, an automatic transmission, front wheel drive, and a 3 liter, 6 cylinder engine. It features a power sun roof, rain sensing wipers, a CD player with MP3 capability, and stabilizers. Call now to take a test drive.” 
         [0043]    The ordered set of images captured of that vehicle with their carefully controlled angle and position are then stitched together with the audio to create audio-video footage. 
         [0044]    In one embodiment an audio/video file containing video is kept as a video track while the automatically generated voiceover is kept as an audio track playing simultaneously. 
         [0045]    Making images for each of a group of products so they all have substantially the same views is accomplished by providing a set of alignment features to a digital imaging device, such as a smart phone. The set of alignment features provided to the digital imaging device may be automatically determined—or at least narrowed down to particular choices that can be made by the photographer—based on the VIN. 
         [0046]    In one embodiment that was created by the present applicants and shown in the flow chart of  FIG. 1   a  the VIN is entered by scanning a barcode on the vehicle with the digital imaging device, as shown in box  40 . The bar code information is transmitted by the digital imaging device to cloud based system  41  run by Dealer Dot Com, Inc., and the bar code information is there decoded, as shown in box  42 . A list of potential matching vehicle types is returned to the digital imaging device, as shown in box  44 . The photographer then selects one vehicle type from the list that matches the actual vehicle, as shown in box  46 . 
         [0047]    Based on that actual vehicle selection, attributes of the actual vehicle then populate a memory for that vehicle in cloud system  41 , as shown in box  48 . A set of alignment features appropriate for the vehicle type so determined is selected on cloud system  41 , as shown in box  50 . The alignment features included in the set by cloud system  41 , and the order in which they are provided, is based on the type of vehicle selected. The set of alignment features may already be in a memory or the digital imaging device. Alternatively the alignment is transmitted to the digital imaging device from cloud system  41  for use in taking photographs of the vehicle with the digital imaging device, as shown in box  51  in  FIGS. 1   a ,  1   b.    
         [0048]    The set of vector based semi-transparent overlay structures appropriate for each vehicle type with the desired sequence of views was created using an image manipulation application, Adobe Fireworks, as described in box  52 . Pixelmator or another such program may also be used. The present applicants used libraries of existing stock photography of cars and trucks to prepare the stencils for the overlay structures. The image angles, perspectives, and zoom levels were determined by the images in these libraries of existing stock photography. Libraries of existing stock photography of cars and trucks are available from vendors such as Evox Images, Rancho Dominguez, Calif. Alternatively, other sources of images, 3D images, and videos can be used as stencils from which to start to determine useful angles or perspectives. 
         [0049]    Export versions of the semi-transparent overlay structures appropriate for each vehicle type with the desired sequence of views were customized for various devices, such as iPad and iphone, which have different aspect ratios, so the overlay structures appropriately fit into the view screen of the digital imaging device as PNG&#39;s or JPG&#39;s or any other image type used by the plugin program running on the digital imaging device, as described in box  53 . The image manipulation tool used, Adobe Fireworks, allows adjusting the size for each type of digital imaging device. Once the overlays were complete, they were loaded into an App for the plugin to use on the digital imaging device. The App may be downloaded from the internet. 
         [0050]    On Apple products, Objective-C is a programming language that has a library that allows creation of a camera view and that allows insertion of a shot profile overlay into the view screen of the digital imaging device. It also allows adjusting the opacity setting so the viewer sees the semi-transparent ghosted view of the overlay. For Android products, Java is a programming language that has similar libraries. For other digital imaging devices, such as windows and Blackberry, other languages and libraries may be used. A similar process is used if the alignment features are made with an overlay outline or with a semi-transparent ghosted view. 
         [0051]    Cloud system  41  runs a program that calls up the first of the set of alignment features for display on the screen of the digital imaging device, as shown in boxes  54  and  56  in  FIG. 1   b.    
         [0052]    After the first of the alignment features is displayed on the screen of the digital imaging device, as shown with ghost semi-transparent overlay structures in  FIGS. 2   a ,  3   a , the first image of the vehicle may be captured. The photographer positions the digital imaging device so that the actual vehicle fits in the ghost semi-transparent overlay structure alignment feature, as shown in box  58  and in  FIGS. 2   b ,  3   b . Positioning the digital imaging device, orientation of the digital imaging device, and focal length of the imaging lens of the digital imaging device may be used to align the actual image of the vehicle on the screen with the alignment feature on the screen. The photographer then captures the image. Capturing the image with the digital imaging device so positioned provides a recorded image of the vehicle defined by the angle, distance, and shape of the alignment feature. The photographer then reviews the image and either accepts or rejects it, as shown in box  60 . If the image is accepted, as shown in box  62 , the App program running on the digital imaging device is notified and the App then repeats the steps for the next alignment feature, as shown in boxes  51  and  54 - 62 . 
         [0053]    In an alternative embodiment, the digital imaging device can be programmed to automatically capture the image when the actual image of the vehicle is correctly positioned on the screen in the alignment feature. This can be accomplished by having the digital imaging device capture a series of images and as each is captured measuring the difference between location of the actual image and the alignment feature and continuing to capture images until the difference around the perimeter of the semi-transparent overlay structure is within a pre-specified tolerance. That captured image is then retained and the others discarded. 
         [0054]    The process sequence of boxes  54  to  62  is repeated for each alignment feature in the set of alignment features until all the alignment features in the set for that vehicle type have been used to capture images. Thus, a set of images of the particular vehicle is created in which each was obtained at a predetermined angle and distance from the car or location in or under the or viewing the engine of the vehicle corresponding to that provided by the alignment features. In this way sets of images of vehicles of the same type have substantially the same views to a high level of correspondence. 
         [0055]    In one embodiment, once all the images corresponding to all the alignment features have been captured and accepted by the photographer, the photographer taps save on the device, the image files are automatically resized, as shown in box  63 , and the image files are uploaded for saving to the dealer inventory and/or to a live website, both provided by system  41 , as shown in box  64 , as shown in  FIGS. 2   a ,  2   b.    
         [0056]    In one embodiment, uploading to system  41  for dealer inventory and/or live website involves a webload manager that adds accepted pictures to a queue locally stored on the device, as shown in boxes  66  and  68 . The App then checks whether the device is connected to the internet, as shown in box  69 . If not, as shown in box  70 , the upload manger waits until the App notifies it that connection is established or re-established, as shown in box  72  and starts or restarts uploading from the queue to the server, as shown in box  74 . Thus, the plurality of images is uploaded from a queue of images while retaining each of the images in the queue until that image has been successfully uploaded. 
         [0057]    Video footage is then created on the website from the images combined with the appropriate audio fragments, as described herein above and in the &#39;279 patent and shown in box  66 . In one embodiment, meta data provided with the images that identify what attribute is shown may be used to direct the order of the audio fragments that are then stitched together. In another embodiment, images showing attributes of the vehicle are ordered according to the sequence provided by the template used for creation of the audio fragments. In either of these embodiments, time for showing each image in the video is determined by the time of the audio fragment used for describing the attribute shown in that photo. 
         [0058]    While several embodiments, together with modifications thereof, have been described in detail herein and illustrated in the accompanying drawings, it will be evident that various further modifications are possible without departing from the scope of the invention as defined in the appended claims. Nothing in the above specification is intended to limit the invention more narrowly than the appended claims. The examples given are intended only to be illustrative rather than exclusive.