GUIDED REAL-TIME VEHICLE IMAGE ANALYZING DIGITAL CAMERA WITH AUTOMATIC PATTERN RECOGNITION AND ENHANCEMENT

Apparatus and associated methods relate to image generation systems. In an illustrative example, an image capturing module (ICM) may be configured to capture a real-time image of a target object (e.g., a car). The ICM, for example, may include a gyro-sensor and a user interface. For example, in a studio mode, the ICM may automatically generate a 360° image of the target object as a function of the gyro-sensor measurement. In a guided capture mode, for example, the ICM may generate guidance indicia at the user interface to assist in capturing components of the target object. In some implementations, the ICM may automatically apply image adjustments based on pixel segmentation and color identification such that the images captured are according to a predetermined image standard profile. Various embodiments may advantageously automatically be captured and standardized images of a target object based on the real-time image.

TECHNICAL FIELD

Various embodiments relate generally to image capture systems having automatically generated guidance with image analysis applications based on color and object classifications.

BACKGROUND

Commercial photography is widely used in wholesale, retail, and professional services. The commercial photography may include advertising photography for illustrating and selling a service or product, architectural photography, event photography for photographing guests and occurrences at mostly social events, fashion photography, etc. In some examples, a commercial photography may include 360-degree product photography that displays a series of photos to give the impression of a rotating object. In some examples, the 360-degree photo may be popular in ecommerce web sites to visualize various products.

Image analysis is an essential component of commercial photography. In order to capture high-quality images, photographers rely on image analysis techniques to determine the best lighting, composition, and other variables that will make their photographs stand out. Image analysis can be performed using a variety of tools, such as digital cameras, image editing software, and specialized algorithms. By leveraging these tools, commercial photographers can produce stunning images that are well-suited to their clients' needs.

In addition to image analysis, commercial photography often involves a variety of image enhancement and transformation techniques. These techniques can be used to adjust the color balance, contrast, and other properties of an image in order to achieve a desired effect. For example, an architectural photographer may use image enhancement techniques to highlight the texture of a building's facade, while a fashion photographer may use transformation techniques to create unique visual effects that enhance the appearance of a model's clothing or accessories. By applying these techniques, commercial photographers can produce images that are both visually stunning and highly effective at conveying a specific message or aesthetic.

SUMMARY

Apparatus and associated methods relate to image generation systems. In an illustrative example, an image capturing module (ICM) may be configured to capture a real-time image of a target object (e.g., a car). The ICM, for example, may include a gyro-sensor and a user interface. For example, in a studio mode, the ICM may automatically generate a 360° image of the target object as a function of the gyro-sensor measurement. In a guided capture mode, for example, the ICM may generate guidance indicia at the user interface to assist in capturing components of the target object. In some implementations, the ICM may automatically apply image adjustments based on pixel segmentation and color identification such that the images captured are according to a predetermined image standard profile. Various embodiments may advantageously automatically be captured and standardized images of a target object based on the real-time image.

Various embodiments may achieve one or more advantages. For example, some embodiments may advantageously identify a vehicle identification number automatically from an image captured by the image capturing device. Some embodiments, for example, may automatically detect a position of the target object from a captured image and advantageously generate guidance at the user interface to automatically guide an adjustment of the position. For example, some embodiments may advantageously stop recording in the studio mode automatically when a full circle is completed. Some embodiments, for example, may advantageously generate a warning in the studio mode once an external disturbance is above a threshold. For example, some embodiments may advantageously detect a position of the target object and automatically adjust a zoom level at the target object based on the position. Some embodiments, for example, may advantageously verify whether a user has captured a required image. For example, some embodiments may automatically apply a color balance filter for an interior image of the target object. Some embodiments, for example, may advantageously remove and replace a background of an image automatically. For example, some embodiment may advantageously remove glare and reflection from a captured image.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

To aid understanding, this document is organized as follows. First, to help introduce discussion of various embodiments, an automatic guidance and image capturing engine (AGICE) is introduced with reference toFIGS.1-2. Second, that introduction leads to a description with reference toFIGS.3A-4Bof some exemplary embodiments of operation modes for the AGICE. Third, with reference toFIGS.5A-C, an exemplary external control device is described in application to exemplary AGICE. Fourth, with reference toFIGS.6-8, this document describes exemplary apparatus and methods useful for using the AGICE. Finally, the document discusses further embodiments, exemplary applications and aspects relating to automatically capturing and standardizing images of a target object.

FIG.1depicts an exemplary automatic guidance and image capturing engine (AGICE) employed in an illustrative use-case scenario. In a vehicle photo-shooting session100, an image capturing device105may be used to capture multiple pictures101of a vehicle110. For example, the image capturing device105may be a mobile device. For example, the image capturing device105may be a digital camera. For example, the multiple pictures101may be used for marketing the vehicle110(e.g., for posting on a seller website, a magazine, a blog).

As shown, the image capturing device105includes an automatic guidance and image capturing engine (AGICE115). In some implementations, the AGICE115may automatically detect an operating mode of the Image capturing device105. In this example, the operating mode includes a studio mode120and a guided capture mode125. For example, the Image capturing device105may selectively operate in the studio mode120or the guided capture mode125by a user130using a user interface of the AGICE115.

As shown, the user130uses the image capturing device105in the studio mode120in a rotating photo-booth135. In some implementations, the rotating photo-booth135may include a rotating wall structure. For example, the rotating wall may spin 360° around an object within the rotating photo-booth135.

In this example, the image capturing device105includes a gyro-sensor140. For example, the gyro-sensor140may generate a gyro-sensor145as a function of a detected angular displacement of the image capturing device105. In the studio mode120, the AGICE115, for example, may receive the gyro-sensor145. As an illustrative example shown inFIG.1without limitation, after turning on the studio mode120, the user130may mount the image capturing device105onto an edge of the wall of the rotating photo-booth135. Then, for example, the user130may push the wall around in a full circle. For example, the AGICE115may generate an angular displacement of image capturing device105based on the measured angular information by the gyro-sensor140. For example, the plurality of images are captured based on an angular displacement between a previously captured image and a current image.

The AGICE115, in some implementations, may include a 360° automotive spin video capture function. In the studio mode120, the AGICE115may load a first objective model150. The first objective model150includes a predetermined angular displacement155. Based on the predetermined angular displacement155, the AGICE115may capture images of the vehicle110periodically. For example, the gyro-sensor140may generate an angular displacement of the image capturing device105. For example, the predetermined angular displacement155may be 3°. For example, the predetermined angular displacement155may be 5°. For example, the predetermined angular displacement155may be 10°. In some implementations, for example, the predetermined angular displacement155may be 15°.

In some implementations, by way of example and not limitation, the predetermined angular displacement155may be 20° or greater. In some implementations, the predetermined angular displacement155may be variable (e.g., depending on location around the circle, depending on the vehicle). The variable predetermined angular displacement may, for example, be linear. In some implementations, the variable predetermined angular displacement may, for example, be non-linear (e.g., depending on color, depending on field of view overlap). For example, in some implementations, the variable angular displacement may be dynamically determined based on predetermined parameters.

Some implementations of the exterior studio mode may, for example, be implemented such as disclosed at least with reference to FIGS. 1-2, 4A, and 5-7, and pages 6-19 of Appendix A, of U.S. Provisional Application Ser. No. 63/364,250, titled “Artificial Intelligence Guided Interactive Vehicle Photograph Guidance and Enhancement on Handheld Device,” filed by Jay Smithweck, on May 5, 2022, the entire contents of which are incorporated herein by reference.

Some implementations of the exterior studio mode may, for example, be implemented such as disclosed at least with reference to FIGS. 1 and 3-4 of U.S. Provisional Application Ser. No. 63/500,190 titled “Artificial Intelligence Guided Interactive Vehicle Photograph Guidance and Enhancement on Handheld Device, filed by Jay Smithweck, et al., on May 4, 2023, the entire contents of which are incorporated herein by reference.

In some implementations, the first objective model150may also include a stabilizing model. Using the gyro-sensor145, the AGICE115may apply the stabilizing model to a capture stream of images by the image capturing device105as a function of the data received from the gyro-sensor140. In some implementations, the AGICE115may automatically stop recording when a full circle is completed. For example, if the predetermined angular displacement155is 5°. The AGICE115may, for example, transmit a stop recording signal to an image capturing module after 72 images are captured.

In some implementations, the stabilizing model may monitor a stream of images captured by the image capturing device105in real-time. For example, the AGICE115may continuously evaluate a recording process in the studio mode120. For example, the AGICE115may be configured to use the first objective model150to ensure that the vehicle110is within view. For example, the AGICE115may use the gyro-sensor145to determine whether the image capturing device105is steady. In some implementations, when excessive vertical movement is detected (e.g., based on the gyro-sensor145) or the vehicle110is captured outside of a predetermined frame (e.g., based on a definition in the first objective model150), the AGICE115may generate an alert to the user. In some examples, the AGICE115may abort the recording to advantageously prevent capturing unusable footage.

In this example, the AGICE115is operably coupled to an image processing server160. For example, the AGICE115may be connected to the image processing server160via a communication network. For example, the AGICE115may be connected to the image processing server160via the Internet. For example, the AGICE115may be connected to the image processing server160via a local area network (LAN). For example, the image processing server160may provide advanced and computationally extensive image processing power to the AGICE115. In some implementations, the AGICE115may use the image processing server160to replace a background of one or more captured images (e.g., the captured video frames in the studio mode120). After the image processing server160processed the captured image, the AGICE115may generate and store the multiple pictures101in a standardized quality.

In some implementations, the image processing server160may include (generative) artificial Intelligence (AI) technology. For example, the AGICE115may use the image processing server160to perform a comprehensive analysis of the captured video frames. For example, the AI technology may isolate the vehicle110from a background wall (e.g., the wall of the rotating photo-booth135) and replace it with a clean, unobtrusive backdrop. In some examples, the AI technology may detect and remove glares of, for example, booth lamps from a vehicle body and windows from the vehicle110. Accordingly, the AGICE115may advantageously provide captured images with standardized visual appearance and professional image.

In the guided capture mode125, the AGICE115may retrieve a second objective model165. For example, the second objective model165may include a predetermined set of photos for capturing an image of the vehicle110. For example, the predetermined set of photos may include various components of the vehicle110(e.g., a front wheel, an engine, a dashboard, a steering wheel). In this example, the second objective model165may include classifier(s)170to identify real-time image parameters from a captured image.

For example, the classifier(s)170may include a segmentation classifier configured to classify an object of the vehicle110(e.g., dashboard) from an image. In some examples, the classifier(s)170may include an object detector configured to generate an output array of bounding boxes. For example, the object detector may generate an output class label for each bounding box. For example, the classifier(s)170may include a pixel segmentation classifier configured to identify pixels in an image that belongs to each of the detected objects (e.g., the dashboard, the back seat, the steering wheel of the vehicle110). In various implementations, some or all of the classifier(s)170may be generated based on trained neural networks. In some implementations, some or all of the classifier(s)170may be generated based on trained decision trees. In some implementations, some or all of the classifier(s)170may be generated based on trained decision trees. In some implementations, some or all of the classifier(s)170may be generated based on trained support vector machines In some implementations, the classifier(s)170may be generated based on a combination of artificial classifiers.

In this example, the AGICE115may generate a guidance175during the studio mode120. For example, the guidance175may include messages and/or images of warning, information, and/or alerts. In some implementations, the classifier(s)170may be used to determine which part of the car has been photographed in the guided capture mode125. As an illustrative example without limitation, after an image is captured, the AGICE115may use the classifier(s)170to check whether a required component is captured. For example, at a step, the user130may be required to take a photo of the steering wheel. However, the user130captured an image of the dashboard. For example, the AGICE115may generate a guidance “are you sure this is of the right image?”

In this example, the image capturing device105further includes an image processing engine (IPE190). For example, the IPE190may generate real-time image data based on a preview image195captured by the image capturing device105. For example, the real-time image data may include real-time image settings (e.g., white balance, zoom, color saturation parameters, contrast). In some implementations, as a function of the real-time image data, the AGICE115may be configured to automatically zoom into a required component at the step. For example, when the current step is to take a photo of the wheel and the wheel is no in a desired frame (e.g., too small, not in center), the AGICE115may use the object detector to detect a location of the wheel in the frame (e.g., around a bounding box), and transmit a signal to the image capturing device to zoom (e.g., focus) into the wheel.

In some implementations, the AGICE115may use the second objective model165to classify and enhance photos based on a current environment and shot type. In some implementations, the classifier(s)170may include a setting classifier to detect whether a photo is taken inside or outside of the rotating photo-booth135and/or at an interior session180or an exterior session185of a photo-shooting session. For example, the AGICE115may detect the interior session180when the user130is taking a photo within the vehicle110(e.g., for the dashboard, the steering wheel). For example, the AGICE115may detect the exterior session185when the user130is taking a photo outside or partially outside of the vehicle110(e.g., for the driver side window control, the engine, the boot).

Based on the detected environment and shot type, the AGICE115may generate corresponding enhanced visual and lighting adjustments for each image. For example, the AGICE115may, in the interior session180, retrieve a color detection model from the second objective model165to inter an interior color of the vehicle110. For example, after a photo is taken, the AGICE115may use the image processing server160to apply predetermined filter(s) based on the inferred interior color.

In the interior session180, for example, the AGICE115may counteract yellowness caused by auto white balance of the image capturing device105. For example, the AGICE115may generate clearer and more vibrant window displays. In some implementations, the AGICE115may also detect a color of the interior seats. Based on the color, the AGICE115may automatically apply an image filter to advantageously complement and accentuate interior aesthetics of the vehicle110.

In some implementations, the AGICE115may automatically apply appropriate lighting filters based on whether the photo shooting session is in the interior session180or the exterior session185. For example, when a photo-session is the exterior session185, the AGICE115may identify a color of the vehicle110and apply color filters (e.g., white balancing filter) as a function of the second objective model165. In some implementations, the AGICE115may perform background replacement, glare removal, floor alteration. For example, the AGICE115may generate shadows and reflections to enhance the overall image quality.

In various implementations, a mobile device (e.g., the image capturing device105) may include an image generation system (e.g., the AGICE115). For example, based on a preview image captured by the mobile device, the image generation system may identify a studio mode and a guided capture mode of a photo-shooting session. For example, in the studio mode, the image generation system may perform studio operations to automatically generate a 360° image of an object (e.g., the vehicle110) based on multiple captured images (e.g., the video recorded while the user130spins the wall of the rotating photo-booth135).

For example, in the guided capture mode, the image generation system may perform classification to identify the object in the preview image. For example, the image generation system may identify an environment of the photo-shooting session (e.g., the exterior session185or the interior session180). For example, the image generation system may identify the object within an object profile (e.g., using the classifier(s)170). For example, the image generation system may generate a guidance output (e.g., the guidance175) at the mobile device.

In some implementations, the image generation system may include a gyroscope (e.g., the gyro-sensor140) configured to detect, in the studio mode, a selected angle of movement of the object (e.g., based on referenced angular movement of the image capturing device105). For example, the mobile device may capture an image of the object based on an angular displacement of the object since the previous image was taken.

In some implementations, the image processing engine may be configured to automatically apply filters to a preview image based on the identified environment to generate a finished image. For example, in the interior session, the image processing engine may infer an interior color of the object, and apply predetermined filter(s) based on the color analysis. In the exterior session, for example, the image processing engine applies a color filter based on the environment.

Some implementations of the guided capture mode may, for example, be implemented such as disclosed at least with reference to FIGS. 1-5 and 8-9, and pages 17-51 of Appendix A, of U.S. Provisional Application Ser. No. 63/364,250, titled “Artificial Intelligence Guided Interactive Vehicle Photograph Guidance and Enhancement on Handheld Device,” filed by Jay Smithweck, on May 5, 2022, the entire contents of which are incorporated herein by reference.

Some implementations of the exterior studio mode may, for example, be implemented such as disclosed at least with reference to FIGS. 1-4, and pages 1-113 of Appendix A, of U.S. Provisional Application Ser. No. 63/500,190, titled “Artificial Intelligence Guided Interactive Vehicle Photograph Guidance and Enhancement on Handheld Device, filed by Jay Smithweck, et al., on May 4, 2023, the entire contents of which are incorporated herein by reference.

In an illustrative example without limitation, the AGICE115may provide additional capabilities in generating marketing pictures for the vehicle110. In an illustrative example, the multiple pictures101are to be generated using a mobile device without the AGICE115, the user130may be required to, first, be trained to memorize all required components for generating the multiple pictures101. Second, the user130may have to manually capture enough images to generate the 360° picture of the vehicle110. Third, the user130may be required to be trained to use photo editing software to process the captured images for background removal/replacement, glare removal, color adjustments, white balance adjustments, or a combination thereof

Using the AGICE115, for example, the user130may be automatically guided, during the exterior session185, to capture a vehicle identification number picture. For example, the VIN picture may be automatically and be optical-character-recognized (OCRed) into the VIN number. For example, the image capturing device105may then automatically retrieve information about the vehicle110based on the VIN number. Thus, the user130may not be required to manually input information to a marketing system.

For example, when the user130starts shooting in the guided capture mode125, the AGICE115may automatically detect a position of a required component of the vehicle110and auto-zoom into the component. For example, the second objective model165may include predetermined picture standards that auto select & apply image filters to make a captured image of the vehicle110attractive to buyers. The second objective model165may also include a predetermined sequence of images to be taken so that the user130may not be required to memorize the sequence. Additionally, the guidance175may be generated to remind the user130to take each of the shots in the sequence.

When the user130is shooting in the interior session180, the AGICE115may determine what kind of colors are inside the vehicle110and apply color filters based on interior (e.g., beige vs gray interior). For example, the applied color filters may override an auto-balance of the image capturing module of the image capturing device105. For example, accordingly, the AGICE115may remove additional work on the photo editing machine for the user130.

For example, when the user130is shooting in the exterior session185, the AGICE115may use the second objective model165and/or the image processing server160to extract the vehicle110from a background (e.g., inside and outside of the rotating photo-booth135). For example, the AGICE115may generate appropriate shallow and light upon replacement of the background. For example, the user130may select from predetermined floors/backgrounds (e.g., white epoxy floor with reflections, gray floor, other colored floors). These operations, if performed manually, may take tens of hours. Additionally, quality of the generated images may be difficult to maintain as uniformly standardized due to human errors.

In some implementations, the AGICE115may include automatic centering and zoom level adjustment. For example, the AGICE115may analyze video frames from the to ensure the car remains centered throughout the spin, automatically adjusting the zoom level for optimal visual representation. Accordingly, the AGICE115may advantageously deliver a consistently high-quality output, irrespective of the user's recording proficiency.

FIG.2is a block diagram depicting an exemplary automatic guidance and image capturing engine (AGICE).FIG.2is a block diagram depicting an exemplary AGICE115. The AGICE115includes a processor205. The processor205may, for example, include one or more processors. The processor205is operably coupled to a communication module210. The communication module210may, for example, include wired communication. The communication module210may, for example, include wireless communication.

In the depicted example, the communication module210is operably coupled to a mounted controller215, a user interface220, and an image capturing device105. For example, the mounted controller215may be releasably coupled to the image capturing device105and configured to transmit image capture signal to the image capturing device105. The image capturing device105, for example, may continuously capture a stream of preview images during operation. For example, upon receiving the image capture signal, the image capturing device105may save a current preview images in a storage device.

The user interface220may, for example, display the stream of preview images from the image capturing device105. For example, the user interface220may display the guidance175. In some examples, the user interface220may also include user control. For example, the user control may include control for the user130to select the operation mode (e.g., the studio mode120, the guided capture mode125) of the AGICE115.

The communication module210is operably coupled to the gyro-sensor140and the image processing server160. For example, the AGICE115may receive the gyro-sensor145from the gyro-sensor140.

For example, the AGICE115may transmit a captured image to the image processing server160to be processed (e.g., for background replacement). For example, the AGICE115may receive the processed image from the160for storage.

The processor205is operably coupled to a memory module230. The memory module230may, for example, include one or more memory modules (e.g., random-access memory (RAM)). The processor205includes a storage module235. The storage module235may, for example, include one or more storage modules (e.g., non-volatile memory).

In the depicted example, the storage module235includes a Session Setup Engine (SSE240). The SSE240may, for example, perform set up operations for a user photo session. In some implementations, the SSE240may identify a target car. For example, the SSE240may identify the interior session180and the exterior session185. In some implementations, the SSE240may identify, based on an image received, detailed information (e.g., a color and make) of the target car. In some implementations, the SSE240may prompt a user input (e.g., an input string) to identify the detailed information of the target car.

The storage module235also includes the IPE190and a guidance engine250. For example, the IPE190may generate real-time image data based on a preview image195captured by the image capturing device105. In some implementations, the IPE190may operate based on a detected session from the SSE240. For example, the IPE190may be operated in the studio mode to automatically capture a 360° photo of the target car based on the predetermined angular displacement155. For example, the IPE190may select and apply different sets of filters based on whether the current photo-session is the interior session180or the exterior session185.

The guidance engine250, for example, may be generated based on user input and a current preview image from the image capturing device105. For example, based on a current step, the guidance engine250may generate an image and a text to remind the user130of which component of the target car is to be taken. Some embodiments of the guidance generated by the guidance engine250are discussed with reference toFIGS.4A-B.

In some implementations, the guidance engine250may generate alerts or warnings to the user based on input from the gyro-sensor140. For example, if a movement of the image capturing device105is above a threshold, the guidance engine250may generate a message to the user that captured images are not usable.

The processor205is operably coupled to a data store255. The data store255includes a mode classification model260and an image standard profile(s) (ISPs265). In various implementations, the AGICE115may retrieve different ISPs265based on an identified model/make of an object. For example, the SSE240may use the mode classification model260to identify an operating mode (e.g., studio mode and guided capture mode, and interior session and exterior session).

The ISPs265includes a capture sequence database270and a background database275. In each operating mode, for example, the AGICE115may retrieve the ISPs265to generate guidance and image filters. For example, based on a current session, the guidance engine250may generate the guidance175to guide the user130for a required component. For example, the IPE190may identify the required component based on the mode classification model260and auto-zoom based on the classifier(s)170of the second objective model165.

The IPE190may include image generative adversarial networks (GANs). For example, using the GANs, the IPE190may generate new images by removing a background of a captured image and replacing it with an image in the background database275.

The data store255also includes the first objective model150and the second objective model165. For example, in the studio mode, the IPE190may retrieve the first objective model150to apply image filters (e.g., white balance filter based on lighting in the rotating photo-booth135) and use the predetermined angular displacement155to automatically capture a full circle image of the target car. In the guided capture mode, the IPE190may use the classifier(s)170to identify components (e.g., Steering wheel, Dashboard, Driver Seat, Rear seats) of the target car.

The ISPs265also includes enhancement filters280. For example, the IPE190may use the enhancement filters280to process a captured image based on a predetermined image standard. In some implementations, the enhancement filters280may also include segmentation models to segment car, wall, floor and window regions. For example, based on the identified segment, the IPE190may apply filters to, for example, remove glare at the windows and add contrast to sharpen a color of the exterior of the car. In some implementations, the IPE190may automatically remove floor, wall, and other unwanted objects (e.g., person, carton boxes) from a captured image.

In various implementations, image processing operations (e.g., application of filters, image setting adjustments) may be performed offline without invoking the image processing server160.

FIG.3AandFIG.3Bdepict an exemplary AGICE operating in a studio mode. As shown inFIG.3A, the image capturing device105may be attached to a mounted controller305(e.g., the mounted controller215as described inFIG.2). In the studio mode120, the image capturing device105may be releasably attached to an edge of a wall of the rotating photo-booth135using the mounted controller305. Various embodiments of the mounted controller305are further described with reference toFIGS.5A-C.

As an illustrative example without limitation, the user130may select a start capture button310. As shown inFIG.3B, the user130may push a wall315after selecting the start capture button310. For example, the AGICE115may automatically record a full circle of the vehicle110. For example, the AGICE115may stop recording when a full circle of the vehicle110is recorded. For example, mounting the image capturing device105on a stable structure (e.g., the wall315) may reduce external disturbance in image capturing during the studio mode.

In some implementations, the AGICE115may also generate guidance in the studio mode. For example, the AGICE115may use a classifier (e.g., the classifier(s)170as described inFIG.1) to detect the vehicle110in a preview image320.

As an illustrative example shown inFIG.3Awithout limitation, based on a position of the vehicle110in the preview image320, the guidance engine250may generate bounding boxes325to guide a user to place the vehicle110in an optimal position for recording.

In some implementations, when an external disturbance (e.g., when the image capturing device105is knocked off by the user, or the user touched the image capturing device105by mistake) is detected during recording in the studio mode, the guidance engine250may generate a warning. In some examples, such warning may be disabled in the guided capture mode because movement is expected.

In some implementations, the first objective model150may include a scene recognition system. In some implementations, the AGICE115may use the SRS to identify front, rear, and side shots of the vehicle110. For example, the AGICE115may use the SRS to automatically record a video sequence. For example, based on the identification, a user may customize the starting position (e.g., start from front of the vehicle110) of a spin video to generate a tailored viewing experience.

FIG.4AandFIG.4Bdepict an exemplary AGICE operating in a guided capture mode. As shown inFIG.4A, a user interface400includes the preview image195and the start capture button310. The user interface400also includes a visual indicia405of a required component of a current shot. In this example, the current shot is 10thof 36 shots, and a required component is a front wheel.

The user interface400also includes a question button410(e.g., a “Cue”). For example, the user130may select the question button410to generate and/or display further guidance from the AGICE115. As shown inFIG.4B, a guidance interface415may be generated when the question button410is selected. As shown, a text guidance420and a guidance image425may be generated based on the current shot of the capture sequence database270. In some implementations, by way of example and not limitation, the question button410may be displayed as a “What's the Focus” button. The What's the Focus button may, for example, provide a user a cue on what the intended subject matter and/or composition of the present image should be focused on (e.g., by the user, automatically).

FIG.5A,FIG.5B, andFIG.5Cdepict an exemplary mounted controller500releasably coupled to an image capturing device having an AGICE. As shown inFIG.5A, an exemplary mounted controller500includes a trigger505. In some implementations, the exemplary mounted controller500may be operably coupled wirelessly (e.g., via a Bluetooth network, a near field contact network, a wireless fidelity network) to the image capturing device105. In some implementations, the exemplary mounted controller500may be operably coupled to the image capturing device105via a wired network (e.g., via a universal serial bus network). For example, when the trigger505is compressed, the exemplary mounted controller500may transmit a signal to the image capturing device105to capture a photo. In some examples, the trigger505may advantageously promote a more stable image compared to selecting the start capture button310.

In this example, the exemplary mounted controller500may be physically adjustable in 360°. As shown inFIGS.5B-C, a body510of the exemplary mounted controller500may be coupled to a coupling element515via a ball joint520. In some implementations, the coupling element515may be configured to be magnetically coupled to the wall315. In some implementations, the coupling element515may be configured to be adhesively coupled to the wall315.

FIG.6depicts an exemplary runtime method600of an exemplary AGICE. For example, the method may be performed by the AGICE115. In this example, the method600begins when a selection signal indicating an operation mode is received in step605. For example, the user130may select an operating mode using the user interface220. In step610, from a data store on the camera, an image standard profile is retrieved. For example, the AGICE115may retrieve the ISPs265. For example, the ISPs265may include the capture sequence database270and the enhancement filters280.

Next, a signal of continuous stream of images is received from a camera in step615. For example, the image capturing device105may continuously update a preview image to the AGICE115. In a decision point620, it is determined whether a studio mode is selected. For example, the user selection in step605indicated that it is a studio mode. In some examples, the AGICE115may use the SSE240and the mode classification model260to determine whether it is a studio mode.

If it is in the studio mode, in step625, a 360° image of the target object is automatically generated. For example, the IPE190may generate the 360° image using images recorded by the image capturing device105. Various methods used in the studio mode are described with reference toFIG.7. Next, an image adjustment based on the image standard profile is applied in step630, and the method600ends. For example, the AGICE115may use the enhancement filters280to the captured image(s) based on a current mode and real-time image analysis parameters (e.g., from the real-time images captured by the image capturing device105).

If, in the decision point620, it is not in the studio mode (e.g., in the guided capture mode), in step635, guidance indicia at a user interface is generated to generate a predetermined sequence of photos for the target object. For example, the AGICE115may use the capture sequence database270to generate the guidance indicia, and the step630is repeated. Various methods used in the guided capture mode are described with reference toFIG.8.

FIG.7depicts an exemplary studio mode image capturing method700. For example, the AGICE115may perform the studio mode image capturing method700. In some implementations, the AGICE115may perform the studio mode image capturing method700in the step625. The method700begins in step705when a target object is detected using an image capturing module. For example, the image capturing device105may be mounted on the wall315facing the vehicle110. For example, the AGICE115may identify the vehicle110using the classifier(s)170from a preview image. In step710, a guidance is generated to center the target object. For example, the guidance engine250may generate a bounding box to center the vehicle110in the preview image.

In step715, a start recording signal is received. For example, the user130may select the start capture button310to generate the start recording signal to the AGICE115. Next, a first objective model including predetermined angular intervals may be retrieved from a data store in step720. For example, the first objective model150including the predetermined angular displacement155is retrieved from the data store255.

In step725, an angular displacement of the image capture module is generated. For example, the AGICE115may receive the angular displacement of the camera from the gyro-sensor140. For example, the AGICE115may determine the angular displacement of the image capture module using the received images from the image capturing device105.

Next, periodic signals are transmitted to the image capture module to capture a plurality of images of the target object as a function of the angular displacement of the image capture module and the predetermined angular intervals in step730.

In decision point735, it is determined whether a full circle of images are captured. For example, the AGICE115may determine that a full circle of images are captured if the predetermined angular displacement155is 5° and 72 pictures are captured around the target object. If the full circle of images are not captured, the step730is repeated.

If the full circle of images are captured, in step740, a signal is transmitted to stop recording. For example, the AGICE115may transmit a signal to the image capturing device105to stop recording. In step745, a background replacement model is applied to captured images. For example, the IPE190may apply the GANs to replace a background of the captured image using a background in the background database275. In step750, position labels are assigned to each of the captured images associated with the angular displacement of the corresponding image, and the method700ends. For example, the AGICE115may assign labels (e.g., front, back, side) to the captured images in the studio mode. For example, based on the labels, a user may customize the starting position (e.g., start from front of the vehicle110) of a spin video generated by the captured images.

FIG.8depicts an exemplary guided capture mode image capturing method800. For example, the method800may be performed in the step635as described with reference toFIG.6. The method800begins in step805when a real-time image is received from an image capturing module. For example, the image capturing device105may capture continuously a stream of real-time images. Next, in step810, a current photo-shooting session is determined to be an exterior session or an interior session. For example, the AGICE115may use the SSE240to determine whether a current photo-shooting session is an exterior session or an interior session. In step815, a second objective is retrieved. For example, the second objective model165may be configured to identify components of a target object. For example, the second objective model165may include the capture sequence database270that may specify an order of target components to be captured in the current photo-shooting session.

After retrieving the second objective mode, a guidance indicia is generated at the user interface as a function of the real-time image and the second objective model in step820. For example, when the capture sequence database270may specify that a front wheel is to be captured, the user interface400may be displayed. In step825, a target component is identified as a function of the predetermined capturing order in the real-time images. For example, the AGICE115may use the classifier(s)170to identify a target component indicated in the capture sequence database270based on a received preview image from the image capturing device105. Next, in step830a control signal is generated to adjust an image setting of the image capturing module based on the real-time image analysis parameters and the real-time image. For example, the AGICE115may generate a control signal to adjust image settings (e.g., white balance and color saturation) based on the ISPs265and identified components using the classifier(s)170.

In step835, a capture signal is received. For example, the AGICE115may receive the capture signal from the start capture button310. For example, the AGICE115may receive the capture signal from the trigger505. In a decision point840, it is determined whether it is an interior session.

If it is an interior session, in step845, an interior color of the target object is identified, and a predetermined filter(s) is applied based on the identified interior color, and the method800ends. For example, the AGICE115may counteract yellowness caused by auto white balance of the image capturing device105. For example, the AGICE115may use the classifier(s)170to detect a color of the interior seats. Based on the detected color, the AGICE115may automatically apply an image filter (e.g., from the enhancement filters280) to advantageously complement and accentuate interior aesthetics of the vehicle110. If it is an exterior session, in step850, a color of an exterior environment of the target object is determined, and a color filter is applied based on the identified color of the exterior environment, and the method800ends. For example, the color filter may include a white balance filter.

Although various embodiments have been described with reference to the figures, other embodiments are possible. For example, the AGICE115may be applied to capture images of other merchandise to generate standardized image profiles for marketing. For example, a target object may be a building (e.g., a house, a hotel room, a trailer home). For example, a target object may be clothing (e.g., a jacket, a dress, a trench-coat). For example, a target object may be a boat. For example, a target object may include an aircraft. For example, a target object may include equipment (e.g., medical equipment, machinery, home appliances).

In various implementations, the methods (e.g., the methods600,700,800), and/or the systems (e.g., the AGICE115) may be implemented in a computer program product. In various implementations, the methods (e.g., the methods600,700,800), and/or the systems (e.g., the AGICE115) may be implemented in a computer-implemented method.

Although an exemplary system has been described with reference toFIG.1, other implementations may be deployed in other industrial, scientific, medical, commercial, and/or residential applications.

In various embodiments, some bypass circuits implementations may be controlled in response to signals from analog or digital components, which may be discrete, integrated, or a combination of each. Some embodiments may include programmed, programmable devices, or some combination thereof (e.g., PLAs, PLDs, ASICs, microcontroller, microprocessor), and may include one or more data stores (e.g., cell, register, block, page) that provide single or multi-level digital data storage capability, and which may be volatile, non-volatile, or some combination thereof. Some control functions may be implemented in hardware, software, firmware, or a combination of any of them.

Computer program products may contain a set of instructions that, when executed by a processor device, cause the processor to perform prescribed functions. These functions may be performed in conjunction with controlled devices in operable communication with the processor. Computer program products, which may include software, may be stored in a data store tangibly embedded on a storage medium, such as an electronic, magnetic, or rotating storage device, and may be fixed or removable (e.g., hard disk, floppy disk, thumb drive, CD, DVD).

Although an example of a system, which may be portable, has been described with reference to the above figures, other implementations may be deployed in other processing applications, such as desktop and networked environments.

Temporary auxiliary energy inputs may be received, for example, from chargeable or single use batteries, which may enable use in portable or remote applications. Some embodiments may operate with other DC voltage sources, such as batteries, for example. Alternating current (AC) inputs, which may be provided, for example from a 50/60 Hz power port, or from a portable electric generator, may be received via a rectifier and appropriate scaling. Provision for AC (e.g., sine wave, square wave, triangular wave) inputs may include a line frequency transformer to provide voltage step-up, voltage step-down, and/or isolation.

Although particular features of an architecture have been described, other features may be incorporated to improve performance. For example, caching (e.g., L1, L2, . . . ) techniques may be used. Random access memory may be included, for example, to provide scratch pad memory and or to load executable code or parameter information stored for use during runtime operations. Other hardware and software may be provided to perform operations, such as network or other communications using one or more protocols, wireless (e.g., infrared) communications, stored operational energy and power supplies (e.g., batteries), switching and/or linear power supply circuits, software maintenance (e.g., self-test, upgrades), and the like. One or more communication interfaces may be provided in support of data storage and related operations.

Some systems may be implemented as a computer system that can be used with various implementations. For example, various implementations may include digital circuitry, analog circuitry, computer hardware, firmware, software, or combinations thereof. Apparatus can be implemented in a computer program product tangibly embodied in an information carrier, e.g., in a machine-readable storage device, for execution by a programmable processor; and methods can be performed by a programmable processor executing a program of instructions to perform functions of various embodiments by operating on input data and generating an output. Various embodiments can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and/or at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

In some implementations, each system may be programmed with the same or similar information and/or initialized with substantially identical information stored in volatile and/or non-volatile memory. For example, one data interface may be configured to perform auto configuration, auto download, and/or auto update functions when coupled to an appropriate host device, such as a desktop computer or a server.

In some implementations, one or more user-interface features may be custom configured to perform specific functions. Various embodiments may be implemented in a computer system that includes a graphical user interface and/or an Internet browser. To provide for interaction with a user, some implementations may be implemented on a computer having a display device. The display device may, for example, include an LED (light-emitting diode) display. In some implementations, a display device may, for example, include a CRT (cathode ray tube). In some implementations, a display device may include, for example, an LCD (liquid crystal display). A display device (e.g., monitor) may, for example, be used for displaying information to the user. Some implementations may, for example, include a keyboard and/or pointing device (e.g., mouse, trackpad, trackball, joystick), such as by which the user can provide input to the computer.

In various implementations, the system may communicate using suitable communication methods, equipment, and techniques. For example, the system may communicate with compatible devices (e.g., devices capable of transferring data to and/or from the system) using point-to-point communication in which a message is transported directly from the source to the receiver over a dedicated physical link (e.g., fiber optic link, point-to-point wiring, daisy-chain). The components of the system may exchange information by any form or medium of analog or digital data communication, including packet-based messages on a communication network. Examples of communication networks include, e.g., a LAN (local area network), a WAN (wide area network), MAN (metropolitan area network), wireless and/or optical networks, the computers and networks forming the Internet, or some combination thereof. Other implementations may transport messages by broadcasting to all or substantially all devices that are coupled together by a communication network, for example, by using omni-directional radio frequency (RF) signals. Still other implementations may transport messages characterized by high directivity, such as RF signals transmitted using directional (i.e., narrow beam) antennas or infrared signals that may optionally be used with focusing optics. Still other implementations are possible using appropriate interfaces and protocols such as, by way of example and not intended to be limiting, USB 2.0, Firewire, ATA/IDE, RS-232, RS-422, RS-485, 802.11 a/b/g, Wi-Fi, Ethernet, IrDA, FDDI (fiber distributed data interface), token-ring networks, multiplexing techniques based on frequency, time, or code division, or some combination thereof. Some implementations may optionally incorporate features such as error checking and correction (ECC) for data integrity, or security measures, such as encryption (e.g., WEP) and password protection.

In various embodiments, the computer system may include Internet of Things (IoT) devices. IoT devices may include objects embedded with electronics, software, sensors, actuators, and network connectivity which enable these objects to collect and exchange data. IoT devices may be in-use with wired or wireless devices by sending data through an interface to another device. IoT devices may collect useful data and then autonomously flow the data between other devices.

Various examples of modules may be implemented using circuitry, including various electronic hardware. By way of example and not limitation, the hardware may include transistors, resistors, capacitors, switches, integrated circuits, other modules, or some combination thereof. In various examples, the modules may include analog logic, digital logic, discrete components, traces and/or memory circuits fabricated on a silicon substrate including various integrated circuits (e.g., FPGAs, ASICs), or some combination thereof. In some embodiments, the module(s) may involve execution of preprogrammed instructions, software executed by a processor, or some combination thereof. For example, various modules may involve both hardware and software.

In an illustrative aspect, an image generation system may, for example, include an image capturing module configured to continuously capture a real-time image; a data store that may, for example, include a program of instructions that may, for example, include an image processing engine configured to generate real-time image analysis parameters as a function of the real-time image, and an image standard profile comprising image enhancement filters.

The image generation system may, for example, includes a user interface operably coupled to the image capturing module, wherein the user interface may, for example, be configured to display an interface including the real-time image; and, a processor operably coupled to the data store such that, when the processor executes the program of instructions, the processor causes operations to be performed to automatically capture and standardized images of a target object based on the real-time image analysis parameters.

The operations may, for example, include: receive a selection signal from the user interface indicating a mode of a photo-shooting session comprising a studio mode and a guided capturing mode, wherein: in the studio mode, perform studio operations to automatically generate a 360° image of the target object, the studio operations that may, for example, retrieve, from a first data store, a first objective model comprising predetermined angular intervals; retrieve a start recording signal; generate an angular displacement of the image capturing module; and, transmit periodic signals to the image capturing module to capture a plurality of images of the target object as a function of the angular displacement of the image capturing module and the predetermined angular intervals such that a plurality of images are taken 360° around the target object; in the guided capture mode, perform guided capture operations to generate guidance indicia at the user interface to generate a predetermined sequence of photos for the target object, wherein the guided capture operations may, for example, include: determine whether a current photo-shooting session is an exterior session or an interior session; retrieve, from a second data store, a second objective model that may, for example, include a pixel segmentation model configured to identify components of the target object, and a predetermined capturing order of target components of the target object for the current photo-shooting session; interactively generate a guidance indicia at the user interface as a function of the real-time image and the second objective model; identify a target component as a function of the predetermined capturing order in the real-time images; and, generate a control signal to adjust an image setting of the image capturing module based on the real-time image analysis parameters and the real-time image; and, apply image adjustments based on the image standard profile, such that the images captured in the studio mode and the guided capture mode may, for example, be captured according to a predetermined image standard profile.

The image generation system may, for example, further include a gyro-sensor configured to measure an angular information of the image capturing module, wherein the image processing engine may, for example, be configured to generate the angular displacement of the image capturing module based on the measured angular information, such that the plurality of images may, for example, be captured based on an angular displacement between a previously captured image and a current image.

The image generation system may, for example, further include a mounting device that may, for example, include a first coupling element configured to releasably couple to the image capturing module, a second coupling element configured to releasably couple to a structure, and a trigger interface, wherein: in the studio mode, the mounting device is mounted on a stable structure such that external disturbance in image capturing is reduced, and, in the guided capturing mode, the mounting device transmits a capture signal to the image when the trigger interface is triggered.

The image generation system may, for example, include, in the studio mode, the studio operations that may, for example, further include: detect the target object; generate a guidance at the user interface to adjust a position of the target object in an image frame; assign a label to a captured image associated with the angular displacement of the captured image; and, generate a warning at the user interface when an external disturbance is above a threshold.

The image generation system may, for example, apply an image adjustment that may, for example, include in the interior session, identify an interior color of the target object, and apply a predetermined filter(s) based on the identified interior color, and, in the exterior session, determine a color of an exterior environment of the target object, and apply a color filter that may, for example, include a white balance filter based on the identified color of the exterior environment.

The image generation system may, for example, include the image processing engine further that may, for example, include a background replacement model configured to selectively replace a background of a captured image.

The image generation system may, for example, generate a control signal to adjust an image setting of the image capturing module that may, for example, automatically adjusts a zoom setting of the image capturing module to focus on the target component.

The image generation system may, for example, include the guidance indicia that may, for example, include an instruction message and a guidance image.

In an illustrative aspect, a computer program product may, for example, include a program of instructions tangibly embodied on a computer readable medium wherein when the instructions are executed on a processor, the processor causes operations to be performed to automatically capture and standardize images of a target object based on real-time image analysis parameters, the operations that may, for example, include: receive, from a user interface of an image capturing module, a selection signal indicating an operation mode that may, for example, include a studio mode and a guided capturing mode; retrieve, from a first data store on the image capturing module, an image standard profile comprising image enhancement filters; receive, from an image capturing module, a signal comprising continuous stream of images; continuously update real-time image analysis parameters that may, for example, include a position of the image capturing module and image settings as a function of the received images; in the studio mode, perform studio operations to automatically generate a 360° image of the target object, the studio operations that may, for example, include: receive a start recording signal; and, generate a plurality of images taken 360° around the target object; in the guided capture mode, perform guided capture operations to generate guidance indicia at the user interface to generate a predetermined sequence of photos for the target object, wherein the guided capture operations that may, for example,: determine whether a current photo-shooting session is an exterior session or an interior session; retrieve, from a second data store, a first objective model that may, for example, include a pixel segmentation model configured to identify components of the target object, and a predetermined capturing order of target components of the target object for the current photo-shooting session; interactively generate a guidance indicia at the user interface as a function of the real-time image and the second objective model; identify a target component as a function of the predetermined capturing order in the real-time images; and, generate a control signal to adjust an image setting of the image capturing module based on the real-time image analysis parameters and the real-time image; apply image adjustments based on the image standard profile, such that the images captured in the studio mode and the guided capture mode that may, for example, be captured according to a predetermined image standard profile.

The computer program may, for example, in the studio mode, the studio operations further: retrieve, from a third data store, a second objective model comprising predetermined angular intervals; generate an angular displacement of the image capturing module based on the real-time image analysis parameters comprising an orientation of the image capturing module; and, transmit periodic signals to the image capturing module to capture the plurality of images of the target object as a function of the angular displacement of the image capturing module and the predetermined angular intervals.

The computer program product may, for example, include the angular displacement of the image capturing module is received from a gyro-sensor.

The computer program product may, for example, in the studio mode, the studio operations further including: detect the target object; generate a guidance at the user interface to adjust a position of the target object in an image frame; automatically generate a signal indicating a full circle is completed, and, generate a warning at the user interface when an external disturbance is above a threshold.

The computer program product may, for example, apply image adjustment including, in the interior session, identify an interior color of the target object, and apply a predetermined filter(s) based on the identified interior color, and, in the exterior session, determine a color of an exterior environment of the target object, and apply a color filter that may, for example, include a white balance filter based on the identified color of the exterior environment.

The computer program product may, for example, include the operations that further may, for example, include: retrieve a third objective model, from a third data store, comprising a background replacement model; and, apply the background replacement model to selectively replace a background of a captured image.

The computer program product may, for example, generate the control signal to adjust an image setting of the image capturing module comprises automatically adjusts a zoom level of the image capturing module to focus on the target component.

The computer program may, for example, include the guidance indicia that may, for example, include an instruction message and a guidance image.

In an illustrative aspect, a computer-implemented method performed by at least one processor to automatically capture and standardized images of a target object, the method may, for example: receive, from a user interface of an image capturing module, a selection signal indicating an operation mode that may, for example, include a studio mode and a guided capturing mode; retrieve, from a first data store on the image capturing module, an image standard profile that may, for example, include image enhancement filters; receive, from an image capturing module, a signal comprising continuous stream of images; continuously update real-time image analysis parameters that may, for example, include a position of the image capturing module and image settings as a function of the received images; in the studio mode, performs studio operations to automatically generate a 360° image of the target object, the studio operations that may, for example, include: retrieve, from a second data store, a first objective model that may, for example, include predetermined angular intervals; generate an angular displacement of the image capturing module based on an angular information received from a gyro-sensor; and, transmit periodic signals to the image capturing module to capture a plurality of images of the target object as a function of the angular displacement of the image capturing module and the predetermined angular intervals such that a plurality of images are taken 360° around the target object; and, in the guided capture mode, the guided capture operations that, may, for example, determine whether a current photo-shooting session is an exterior session or an interior session; retrieve, from a third data store, a second objective model comprising a pixel segmentation model configured to identify components of the target object based on, and a predetermined capturing order of target components of the target object for the current photo-shooting session; interactively generate a guidance indicia at the user interface as a function of the real-time image and the second objective model; identify a target component as a function of the predetermined capturing order in the real-time images; and, generate a control signal to adjust an image setting of the image capturing module based on the real-time image analysis parameters and the real-time image; and, apply image adjustments based on the image standard profile, such that the images captured in the studio mode and the guided capture mode that may, for example, captured according to a predetermined image standard profile.

The computer-implemented method may, for example, include in the studio mode, the studio operations which may, for example, further: detect the target object; generate a guidance at the user interface to adjust a position of the target object in an image frame; automatically generate a signal indicating a full circle is completed, and, generate a warning to the user when an external disturbance that may, for example, be above a threshold.

The computer-implemented method may, for example, apply image adjustment that may, for example, include, in the interior session, identify an interior color of the target object, and apply a predetermined filter(s) based on the identified interior color, and, in the exterior session, determine a color of an exterior environment of the target object, and apply a color filter comprising a white balance filter based on the identified color of the exterior environment.

The computer-implemented method may, for example, further apply a background replacement model configured to selectively replace a background to images captured in the studio mode.