Electronic device and method for processing image

An electronic device includes a first image sensor generating first image data, a second image sensor generating second image data, at least one processor processing the first image data and the second image data, and a display displaying at least one image of the first image data and the second image data processed by the at least one processor, wherein the electronic device is configured to set each time stamp on the first image data and the second image data and the display is configured to display at least one image data on the basis of the time stamps. An operating method of an electronic device includes generating first image data and second image data by using a first image sensor and a second image sensor, respectively, and displaying at least one image data on a display on the basis of the time stamp.

TECHNICAL FIELD

Various embodiments of the present disclosure relates generally to a method and an electronic device for processing an image.

BACKGROUND

With the developments of information and communication technology and semiconductor technology, various electronic devices become multimedia devices providing various multimedia services. For example, a portable electronic device may provide various multimedia services such as broadcasting services, wireless Internet service, and music playback service.

An electronic device may provide various services through at least one image obtained from image sensors.

SUMMARY

If including a plurality of image sensors an electronic device may not display images obtained through the plurality of image sensors on a display unit at the same time due to different processing times of the images.

To address the above-discussed deficiencies, it is a primary object to provide a device and method for efficiently processing images obtained through a plurality of image sensors in an electronic device.

Another object of the present disclosure is to provide a device and method for reducing processing delay of images obtained through a plurality of image sensors in an electronic device.

Another object of the present disclosure is to provide a device and method for processing images obtained through a plurality of image sensors in an electronic device by using image processing units disposed inside or outside of a processor.

Another object of the present disclosure is to provide a device and method for converting at least one image obtained through at least one image sensor into an image in a format displayable on a display unit by using at least one image processor disposed outside a processor in an electronic device.

Another object of the present disclosure is to provide a device and method for setting a time stamp for images obtained through a plurality of image sensors in an electronic device.

Another object of the present disclosure is to provide a device and method for selecting a plurality of images for synthesis by using a time stamp set for images obtained through a plurality of image sensors in an electronic device.

Another object of the present disclosure is to provide a device and method for selecting a plurality of images for synthesis by using a time stamp and an image processing delay time set for images obtained through a plurality of image sensors in an electronic device.

According to an aspect of the present disclosure, an electronic device includes a first image sensor generating first image data, a second image sensor generating second image data, at least one processor processing at least one image data of the first image data and the second image data; and a display unit displaying at least one image of the first image data and the second image data processed by the at least one processor, wherein the electronic device sets a time stamp on the first image data and the second image data and the display unit displays at least one image data on the basis of the time stamp.

According to an aspect of the present disclosure, an electronic device includes a first image sensor generating first image data, a second image sensor generating second image data, an application processor processing the first image data, and an image processing processor processing the second image data, wherein the image processing processor converts the second image data into a format displayable on a display unit.

According to an aspect of the present disclosure, an electronic device includes a storage device storing first image data having a first time stamp added and second image data having a second time stamp added, at least one processor processing at least one image data of the first image data and the second image data, and a display unit displaying at least one image of the first image data and the second data processed by the at least one processor, wherein the display unit displays at least one image data on the basis of the time stamp.

According to an aspect of the present disclosure, an electronic device includes a first image sensor generating first image data; a second image sensor generating second image data, at least one processor processing at least one of the first image data and the second image data, and a storage device storing time stamps corresponding to the first image data and the second image data, wherein the electronic device compares time stamps of the image data.

According to an aspect of the present disclosure, an operating method of an electronic device includes generating a plurality of image data by using a plurality of image sensors, setting a time stamp corresponding to each of the plurality of image data, and displaying at least one image data on a display unit on the basis of the time stamp.

According to an aspect of the present disclosure, an operating method of an electronic device includes generating first image data and second image data by using a first image sensor and a second image sensor, and processing the first image data by using an application processor and processing the second image data by using an image processing processor, wherein the processing of the second image data includes converting the second image data into a format displayable on a display unit.

According to an aspect of the present disclosure, an operating method of an electronic device includes storing first image data including a first time stamp added and second image data including a second time stamp added, and displaying at least one image data on a display unit on the basis of the time stamp.

According to an aspect of the present disclosure, an operating method of an electronic device includes generating first image data and second image data by using a first image sensor and a second image sensor, storing a first image data including a first time stamp added and a second image data including a second time stamp added, and displaying at least one image data on a display unit on the basis of the time stamp.

DETAILED DESCRIPTION

FIGS. 1 through 13, discussed below, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged electronic devices. Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. In the following description, detailed descriptions of well-known functions or constructions will be omitted since they would obscure the disclosure in unnecessary detail. Also, the terms used herein are defined according to the functions of the present disclosure. Thus, the terms may vary depending on user's or operator's intentions or practices. Therefore, the terms used herein must be understood based on the descriptions made herein.

Various embodiments of the present disclosure describe a device and method for efficiently processing images obtained through a plurality of image sensors in an electronic device.

An electronic device according to an embodiment of the present disclosure may be one or a combination of various devices such as smart phones, tablet PCs, mobile phones, video phones, e-book readers, desktop PCs, laptop PCs, netbook computers, PDAs, PMPs, MP3 players, mobile medical equipment, jewelry, electronics accessory, cameras, wearable devices, electronic watches, wrist watches, refrigerators, air conditioners, vacuum cleaners, artificial intelligence robots, TVs, DVD players, audios, ovens, microwaves, washing machines, microwave bracelets, electronic necklaces, air purifiers, electronic picture frames, medical devices (e.g., MRAs, MRIs, CTs, camcorders, or ultrasound devices), navigation devices, GPS receivers, EDRs, FDRs, set-top boxes, TV boxes (for example, SamSung HomeSync™, Apple TVTM™, or Google TVTM™), Electronic dictionaries, automotive infotainment devices, marine electronic equipment (for example, marine navigation systems and gyro compasses), Avionics, security devices, and electronic garments, electronic keys, camcorders, game consoles, HMDs, flat panel display devices, the electronic albums, part of furniture or buildings/structures including an electronic device, electronic boards, electronic signature input devices or projectors, all of which include a plurality of sensors. It is apparent to those skilled in the art that the electronic device is not limited to the above-mentioned devices.

FIG. 1is a block diagram illustrating a configuration of an electronic device according to various embodiments of the present disclosure.

Referring toFIG. 1, the electronic device100includes a processor110, a memory120, image sensors130-1to130-N, a display unit140, and a user input unit150. Here, the processor110can include an application program processor AP.

The processor110can control the electronic device100so that it can provide various services.

The processor110interprets a command received from at least another component (for example, the memory120, the image sensors130-1to130-N, the display unit140, and the user input unit150) included in the electronic device100and then and performs operations and data processing in response to the interpreted command. For example, the processor110can perform at least one image processing on images provided from the image sensors130-1to130-N, for example, level adjustment, noise reduction, gamma correction, and format conversion for displaying an image on the display unit140. The processor110can store the processed images in the memory120or display them on the display unit140. At this point, the processor110can transmit the images displayed on the display unit140to the memory120so as to temporarily store them in the memory120. Here, the image processing, that is, the format conversion for displaying an image on the display unit140, can include color space conversion.

The processor110can allow the electronic device100to provide various multimedia services by executing at least one program stored in the memory120. The processor110can select at least two from among images obtained through the image sensors130-1to130-N and synthesize them by executing a program stored in the memory120. For example, the processor110can set a time stamp corresponding to images obtained through at least one among the image sensors130-1to130-N. As another example, if a capture event occurs, the processor110can select and synthesize at least two images for synthesis on the basis of a time stamp of images obtained through each of the image sensors130-1to130-N. As another example, if a capture event occurs, the processor110can select and synthesize at least two images for synthesis on the basis of a processing delay time and a time stamp for images obtained through each of the image sensors130-1to130-N.

The memory120can store commands or data received or generated from at least one component included in the electronic device100. For example, the memory120can include programming modules, for example, a kernel, a middleware, an application programming interface (API), and an application. Here, the programming module can include an operating system (OS) controlling resources relating to the electronic device100or various applications running on OS. At this point, each programming module can be configured with software, firmware, hardware, or a combination thereof. The OS can include Android, iOS, Windows, Symbian, Tizen, or bada.

The image sensors130-1to130-N can provide collected images obtained through capturing a subject to the processor110. At this point, the image sensors130-1to130-N can transmit an image to the processor110through a serial interface such as MIPI and MDDI or a parallel interface such as parallel bus. Here, the first image sensor130-1can be disposed at the front of the electronic device100and the Nth image sensor130-N can be disposed at the rear of the electronic device100.

The display unit140can provide a graphic user interface such as state information, characters entered by a user, a moving image, or a still image of the electronic device100. For example, the display unit140can display at least one image provided from the processor110. As another example, the display unit140can display at least two images selected by the processor110based on a time stamp or a time stamp and an image processing delay time.

The user input unit150can transmit commands or data generated by a user selection to the processor120or the memory120. For example, the user input unit150can include a touch input unit, a pen sensor, a key, or an ultrasonic input device.

Although not shown in the drawings, the electronic device100can further include a communication unit for communicating with another electronic device or a server through voice communication or data communication. Here, the communication unit can include a plurality of communication sub modules supporting different communication networks. For example, the communication network is not limited thereto but can support a short range communication protocol (for example, Wifi, BT, NFC) or a network communication (for example, Internet, LAN, WAN, telecommunication network, cellular network, satellite network, or POTS).

According to the above-mentioned embodiment, the electronic device100includes the image sensors130-1to130-N. At this point, at least one image sensor among the image sensors130-1to130-N can be selectively mounted on the electronic device100. For example, at least one image sensor among the image sensors130-1to130-N can be selectively mounted on the electronic device100through a wired interface. As another example, at least one image sensor among the image sensors130-1to130-N can be selectively connected to the electronic device100through a wireless interface such as Bluetooth and wireless LAN.

FIG. 2is a block diagram illustrating a configuration of a processor according to various embodiments of the present disclosure.

Referring toFIG. 2, the processor110includes an image processing unit200, a display control unit210, and an image generation control unit220.

The image processing unit200can perform at least one image processing on image data provided from each of the image sensors130-1to130-N, for example, level adjustment, noise reduction, gamma correction, and color space conversion. The image processing unit200can transmit the processed image to at least one of the memory120and the display control unit210. At this point, the image processing unit200can transmit the images displayed on the display unit140to the memory120so as to temporarily store them in the memory120.

The display control unit210can provide a graphic user interface through the display unit140. For example, the display control unit210can display images provided from the image processing unit200or the memory120on the display unit140. At this point, the display control unit210can display images provided from the image sensors130-1to130-N through the image processing unit200on the display unit140simultaneously.

The image generation control unit220can select at least two images from among images obtained through the image sensors130-1to130-N and synthesize them. For example, if a capture event occurs, the image generation control unit220can select and synthesize at least two images for synthesis on the basis of a time stamp of images stored in the memory120. For example, if different sizes of images obtained through the low-pixel first image sensor130-1and the high-pixel Nth image sensor130-N are synthesized, the image generation control unit220can select at least one first image including a time stamp prior to a capture event occurrence time from among images obtained through the first image sensor130-1. At this point, the image generation control unit220can select at least one first image from among images including a time stamp prior to a capture event occurrence time by using a processing delay difference on an image of the first image sensor130-1and an image of the Nth image sensor130-N. The image generation control unit220can select at least one second image obtained at a time closest to the capture event occurrence time according to the time stamp of images obtained through the Nth image sensor130-N and then can synthesize the second image with the first image. At this point, the image generation control unit220can transmit first and second images stored in the memory120to the control unit210so as to display a synthesized image on the display unit140.

Although not shown in the drawings, the processor110can further include a time setting unit for setting a time stamp for at least one image data provided from the image sensors130-1to130-N. For example, the time setting unit can record a time corresponding to each image data provided from the image sensors130-1to130-N by each frame unit. As another example, if there is at least one image sensor that is selectively mounted on the electronic device100among the image sensors130-1to130-N, the time setting unit can set a time stamp on at least one image data provided from the at least one image sensor mounted on the electronic device100. At this point, an image obtained through at least one image sensor that is selectively mounted on the electronic device100can be set with a time stamp by an additional module included in each image sensor.

According to the above-mentioned embodiment, the processor110can process images provided from the image sensors130-1to130-N through the image processing unit200.

According to another embodiment, the processor110can process images provided from the image sensors130-1to130-N through a plurality of image processing units in the processor110.

FIG. 3is a block diagram illustrating a configuration of an electronic device according to various embodiments of the present disclosure.

Referring toFIG. 3, the electronic device300includes a processor310, a memory320, image sensors330-1to330-N, external image processing units340-1to340-(N−1), a user input unit350, and a display unit360. Here, the processor310can include an application program processor AP.

The processor310can control the electronic device300so that it can provide various services.

The processor310interprets a command received from at least another component (for example, the memory320, the first image sensor330-1, the external image processing units340-1to340-(N−1), the user input unit350, and the display unit360) included in the electronic device300and then and performs operations and data processing in response to the interpreted command. For example, the processor310can perform at least one image processing on images provided from the first image sensor330-1, for example, level adjustment, noise reduction, gamma correction, and format conversion for displaying an image on the display unit360. The processor310can store the processed images in the memory320or display them on the display unit360. At this point, the processor310can transmit the images displayed on the display unit360to the memory320so as to temporarily store them in the memory320. As another example, the processor310can convert images stored in the memory320into images in a format for displaying the images on the display unit560through the external image processing units340-1to340-(N−1) and then display the converted images on the display unit360. Here, the image processing, that is, the format conversion for displaying an image on the display unit360, can include color space conversion.

The processor310can allow the electronic device300to provide various multimedia services by executing at least one program stored in the memory320. The processor310can select at least two images from among images obtained through the image sensors330-1to330-N and synthesize them by executing a program stored in the memory320. For example, the processor310can set a time stamp on image data provided from the first image sensor330-1or the first image sensor330-1and the external image processing units340-1to340-(N−1). As another example, if a capture event occurs, the processor310can select and synthesize at least two images for synthesis on the basis of a time stamp of images obtained through each of the image sensors330-1to330-N. As another example, if a capture event occurs, the processor310can select and synthesize at least two images for synthesis on the basis of a processing delay time and a time stamp for images obtained through each of the image sensors330-1to330-N.

The memory320can store commands or data received or generated from at least one component included in the electronic device300.

The image sensors330-1to330-N can provide collected images obtained through capturing a subject to the processor310. At this point, the image sensors330-1to330-N can transmit an image to the processor310or the external image processing units340-1to340-(N−1) through a serial interface such as MIPI and MDDI or a parallel interface such as parallel bus. Here, the first image sensor330-1can be disposed at the front of the electronic device300and the Nth image sensor330-N can be disposed at the rear of the electronic device300.

The external image processing units340-1to340-(N−1) can perform image processing on an image provided from the image sensors330-2to330-N, for example, level adjustment, noise reduction, and gamma correction.

The user input unit350can transmit commands or data generated by a user selection to the processor310or the memory320. For example, the user input unit350can include a touch input unit, a pen sensor, a key, or an ultrasonic input device.

The display unit360can provide a graphic user interface such as state information, characters entered by a user, a moving image, or a still image of the electronic device300. For example, the display unit360can display at least one image provided from the processor310. As another example, the display unit360can display at least two images selected by the processor310based on a time stamp or a time stamp and an image processing delay time.

Although not shown in the drawings, the electronic device300can further include a communication unit for communicating with another electronic device or a server through voice communication or data communication. Here, the communication unit can include a plurality of communication sub modules supporting different communication networks.

According to the above-mentioned embodiment, the electronic device300includes the image sensors330-1to330-N. At this point, at least one image sensor among the image sensors330-1to330-N can be selectively mounted on the electronic device300. For example, at least one image sensor among the image sensors330-1to330-N can be selectively mounted on the electronic device300through a wired interface. In this case, an external image processing unit connected to at least one image sensor that is selectively mounted on the electronic device300can be mounted on the electronic device300or can be selectively mounted on the electronic device300in addition to an Nth image sensor.

As another example, at least one image sensor among the image sensors330-1to330-N can be selectively connected to the electronic device300through a wireless interface such as Bluetooth and wireless LAN. In this case, an external image processing unit connected to at least one image sensor that is selectively connected to the electronic device300can be mounted on the electronic device300or can be selectively mounted on the electronic device300in addition to an Nth image sensor.

FIG. 4is a block diagram illustrating a configuration of a processor according to various embodiments of the present disclosure.

Referring toFIG. 4, the processor310includes an image processing unit400, an internal interface410, a display control unit420, a format conversion unit430, and an image generation control unit440.

The image processing unit400can perform at least one image processing on image data provided from the first image sensor130-1, for example, level adjustment, noise reduction, gamma correction, and color space conversion. The image processing unit400can transmit the processed image to at least one of the memory320and the display control unit420. At this point, the image processing unit400can transmit the images displayed on the display unit360to the memory320so as to temporarily store them in the memory320.

The internal interface410can transmit images provided from each of the external image processing units340-1to340-(N−1) to the memory320. For example, the internal interface410can include at least one of MIFI and CAMIF.

The display control unit420can provide a graphic user interface through the display unit360. For example, the display control unit420can display images provided from the image processing unit400or the memory320on the display unit360. At this point, the display control unit420can display an image provided from the first image sensor330-1through the image processing unit400and an image provided from the Nth image sensor330-N through the memory320on the display unit360simultaneously. For example, the display control unit420can display an image converted into a data format for displaying the image on the display unit360through the format conversion unit430and an image provided from the first image sensor330-1through the image processing unit400simultaneously.

The format conversion unit430can convert an image provided from the memory320into a data format displayable on the display unit360. For example, the format conversion unit430can perform color space conversion on an image provided from the memory320and transmit the converted image to the display control unit420.

The image generation control unit440can select at least two images from among images obtained through the image sensors330-1to330-N and synthesize them. For example, if a capture event occurs, the image generation control unit440can select and synthesize at least two images for synthesis on the basis of a time stamp of images stored in the memory320. If different sizes of images obtained through the low-pixel first image sensor330-1and the high-pixel Nth image sensor330-N are synthesized, the image generation control unit440can select at least one first image including a time stamp prior to a capture event occurrence time from among images obtained through the first image sensor330-1. At this point, the image generation control unit440can select at least one first image from among images including a time stamp prior to a capture event occurrence time by using a processing delay difference on an image of the first image sensor330-1and an image of the Nth image sensor330-N. The image generation control unit440can select at least one second image obtained at a time closest to the capture event occurrence time according to the time stamp of images obtained through the Nth image sensor330-N and then can synthesize the second image with the first image. At this point, the image generation control unit440can transmit first and second images stored in the memory320to the control unit420so as to display a synthesized image on the display unit360.

Although not shown in the drawings, the processor310can further include a time setting unit for setting a time stamp on image data provided from the first image sensor330-1or the first image sensor330-1and the external image processing units340-1to340-(N−1). For example, the time setting unit can record a time corresponding to each image data provided from the first image sensor330-1by each frame unit. At this point, an image obtained through the second image sensor330-2to the Nth image sensor330-N can be set with a time stamp through an external image processing unit connected to each image sensor.

FIG. 5is a block diagram illustrating a configuration of an electronic device according to various embodiments of the present disclosure.

Referring toFIG. 5, the electronic device500includes a processor510, memories520and550, image sensors530-1to530-N, external image processing units540-1to540-(N−1), a display unit560, and a user input unit570. Here, the processor510can include an application program processor AP.

The processor510can control the electronic device500so that it can provide various services.

The processor510can interpret a command received from at least another component included in the electronic device500and performs operations or data processing in response to the interpreted command. For example, the processor510can perform at least one image processing on images provided from the first image sensor530-1, for example, level adjustment, noise reduction, gamma correction, and format conversion for displaying an image on the display unit560. The processor510can store the processed images in the first memory520or display them on the display unit560. At this point, the processor510can transmit the images displayed on the display unit560to the first memory520so as to temporarily store them in the first memory520. As another example, the processor510can convert images stored in the first memory520into images in a format for displaying the images on the display unit560through the external image processing units540-1to540-(N−1) and then display the converted images on the display unit560. Here, the image processing, that is, the format conversion for displaying an image on the display unit560, can include color space conversion.

The processor510can allow the electronic device500to provide various multimedia services by executing at least one program stored in the first memory520. The processor510can select at least two images from among images obtained through the image sensors530-1to530-N and synthesize them by executing a program stored in the memory520. For example, the processor510can set a time stamp on image data provided from the first image sensor530-1or the first image sensor530-1and the external image processing units540-1to540-(N−1). As another example, if a capture event occurs, the processor510can select and synthesize at least two images for synthesis on the basis of a time stamp of images obtained through each of the image sensors530-1to530-N. As another example, if a capture event occurs, the processor510can select and synthesize at least two images for synthesis on the basis of a processing delay time and a time stamp for images obtained through each of the image sensors530-1and530-N.

The first memory520can store commands or data received or generated from at least one component included in the electronic device500.

The image sensors530-1to530-N can provide collected images obtained through capturing a subject to the processor510. At this point, the image sensors530-1to530-N can transmit an image to the processor510or the external image processing units540-1to540-(N−1) through a serial interface such as MIPI and MDDI or a parallel interface such as parallel bus. Here, the first image sensor530-1can be disposed at the front of the electronic device500and the Nth image sensor530-N can be disposed at the rear of the electronic device500.

The external image processing units540-1to540-(N−1) can perform image processing on an image provided from the image sensors530-2to530-N, for example, level adjustment, noise reduction, gamma correction, and color space conversion, and store the processed image in the first memory520. Additionally, the external image processing units540-1to540-(N−1) can set time information in an image provided from the image sensors530-2to530-N and store the image in the second memory550.

The second memory550can store an unprocessed image provided from the external image processing units540-1to540-(N−1). For example, the second memory550can store raw image data provided from the external image processing units540-1to540-(N−1). At this point, the second memory550can exist at each of the external image processing units540-1and540-(N−1).

The display unit560can provide a graphic user interface such as state information, characters entered by a user, a moving image, or a still image of the electronic device500. For example, the display unit560can display at least one image provided from the processor510. As another example, the display unit560can display at least two images selected by the processor510based on a time stamp or a time stamp and an image processing delay time.

The user input unit570can transmit commands or data generated by a user selection to the processor510or the first memory520. For example, the user input unit570can include a touch input unit, a pen sensor, a key, or an ultrasonic input device.

Although not shown in the drawings, the electronic device500can further include a communication unit for communicating with another electronic device or a server through voice communication or data communication. Here, the communication unit can include a plurality of communication sub modules supporting different communication networks.

According to the above-mentioned embodiment, the electronic device500includes the image sensors530-1to530-N. At this point, at least one image sensor among the image sensors530-1to530-N can be selectively mounted on the electronic device500. For example, at least one image sensor among the image sensors530-1to530-N can be selectively mounted on the electronic device500through a wired interface. In this case, an external image processing unit connected to at least one image sensor that is selectively mounted on the electronic device300can be mounted on the electronic device500or can be selectively mounted on the electronic device300in addition to an Nth image sensor.

As another example, at least one image sensor among the image sensors530-1to530-N can be selectively connected to the electronic device500through a wireless interface such as Bluetooth and wireless LAN. In this case, an external image processing unit connected to at least one image sensor that is selectively connected to the electronic device500can be mounted on the electronic device500or can be selectively connected to the electronic device300in addition to an Nth image sensor.

FIG. 6is a block diagram illustrating a configuration of a processor according to various embodiments of the present disclosure.

Referring toFIG. 6, the processor510includes an image processing unit600, an internal interface610, a display control unit620, and an image generation control unit630.

The image processing unit600can perform at least one image processing on image data provided from the first image sensor530-1, for example, level adjustment, noise reduction, gamma correction, and color space conversion. The image processing unit600can transmit the processed image to at least one of the first memory520and the display control unit620. At this point, the image processing unit600can transmit the images displayed on the display unit560to the first memory520so as to temporarily store them in the first memory520.

The internal interface610can transmit images provided from each of the external image processing units540-1to540-(N−1) to the first memory520. For example, the internal interface610can include at least one of MIFI and CAMIF and a raw data dumping interface (RDI) for transmitting an image converted for displaying an image on the display unit560from the external image processing units540-1to540-(N−1).

The display control unit620can provide a graphic user interface through the display unit560. For example, the display control unit620can display images provided from the image processing unit600or the first memory520on the display unit560. At this point, the display control unit620can display an image provided from the first image sensor630-1through the image processing unit600and an image provided from the Nth image sensor530-N through the first memory520on the display unit560simultaneously.

The image generation control unit630can select at least two images from among images obtained through the image sensors530-1to530-N and synthesize them. For example, if a capture event occurs, the image generation control unit630can select and synthesize at least two images for synthesis on the basis of a time stamp of images stored in the first memory520and the second memory550. If images obtained through the low-pixel first image sensor530-1and the high-pixel Nth image sensor530-N are synthesized, the image generation control unit630can select at least one first image including a time stamp prior to a capture event occurrence time from among images stored in the first memory520and obtained through the first image sensor530-1. At this point, the image generation control unit630can select at least one first image from among images including a time stamp prior to a capture event occurrence time by using a processing delay difference on an image of the first image sensor530-1and an image of the Nth image sensor530-N. The image generation control unit630can select a second image obtained at a time closest to the capture event occurrence time according to the time stamp of images stored in the second memory550and obtained through the Nth image sensor530-N and then can synthesize the second image with the first image. As another embodiment, if images obtained through the low-pixel first image sensor530-1and the high-pixel Nth image sensor530-N are synthesized, the image generation control unit630can select at least one second image obtained at a time closest to a capture event occurrence time according to a time stamp of images stored in the second memory550and obtained through the Nth image sensor530-N. The second image stored in the second memory550selected from the image generation control unit630can be stored in the first memory520through the processor510. The image generation control unit630can select a first image obtained at a time closest to a time at which the second image starts to be transmitted to the processor510or at a time closest to a time at which the second image starts to be stored in the first memory520according to the time stamp of images stored in the first memory520and obtained through the first image sensor530-1, and then can synthesize the selected first image with the second image. At this point, the image generation control unit630can transmit first and second images stored in the first memory520to the control unit620so as to display a synthesized image on the display unit560.

Although not shown in the drawings, the processor510can further include a time setting unit for setting a time stamp on image data provided from the first image sensor530-1or the first image sensor530-1and the external image processing units540-1to540-(N−1). For example, the time setting unit can record a time corresponding to each image data provided from the first image sensor530-1by each frame unit. At this point, an image obtained through the second image sensor530-2to the Nth image sensor530-N can be set with a time stamp through an external image processing unit connected to each image sensor. As another example, the time setting unit can record a time corresponding to image data provided from the external image sensor540-(N−1) by each frame unit. For example, the image generation control unit630can select and synthesize at least two images for synthesis on the basis of a time stamp of images stored in the first memory520.

FIG. 7is a block diagram illustrating a configuration of an external image processing unit according to various embodiments of the present disclosure.

Referring toFIG. 7, the external image processing unit540includes an image processing control unit700and a time setting unit710.

The image processing control unit700can perform at least one image processing on image data provided from the image sensors530-2to530-N, for example, level adjustment, noise reduction, gamma correction, and format conversion for displaying an image on the display unit560. For example, the image processing control unit700can convert image data of YUV422 provided from the image sensors530-2to530-N into image data of YUV420 through color space conversion, so as to convert an image into a format displayable on the display unit560.

The image processing control unit700can convert at least one image data stored in the second memory550into a format displayable on the display unit560and then can transmit the converted image into the image generation control unit630. For example, the image processing control unit700can receive selected image data for image synthesis from the second memory550according to a control of the image generation control unit630ofFIG. 6and then can convert the received image data into a format displayable on the display unit560to transmit it to the image generation control unit630. As another example, when a capture event occurs, the image processing control unit700can convert at least one image data among images stored in the second memory550into a format displayable on the display unit560and then can transmit the converted image data into the image generation control unit630.

The time setting unit710can set a time stamp in image data provided from the image sensors530-2to530-N. For example, the time setting unit can include a time insertion unit and a frame setting unit and can record a time corresponding to each image data provided from the image sensors530-2to530-N by each frame unit.

According to the above-mentioned embodiment, the external image processing unit can include an image processing control unit700and a time setting unit710. According to another embodiment, the time setting unit710can be disposed outside an external image processing unit.

FIG. 8is a block diagram illustrating a configuration of a memory according to various embodiments of the present disclosure.

Referring toFIG. 8, the first memory520includes a plurality of blocks800,810, and820logically or physically and stores data therein. For example, image data provided from the image processing unit600of the processor510can be stored in the third block820of the first memory520.

Image data provided from the external image processing units540-1to540-(N−1) in the first block800of the first memory520. At this point, the image data can be classified into Y data, UV data, and metadata and can be stored in the internal blocks802,804, and806of the first block800. Here, the metadata can include at least one of a frame identifier of image data, a time stamp, focus information, and image setting information EXIF.

If a capture event occurs, image data stored in the second memory550can be stored in the third block820of the first memory520through the external image processing units540-1to540-(N−1).

As mentioned above, the electronic device500can convert images obtained through the image sensor530-N into a format for displaying the images on the display unit560through the external image processing unit540-(N−1) and then can store the converted images in the first memory520, so that delay due to an additional format conversion can be reduced.

FIG. 9is a flowchart illustrating a method of selecting images for synthesis in an electronic device according to various embodiments of the present disclosure.

Referring toFIG. 9, the electronic device can display images obtained through a plurality of image sensors on a display unit in operation901. At this point, the electronic device can store at least one image displayed on a display unit in at least one memory. For example, referring toFIG. 5, the electronic device500performs image processing on an image obtained through the first image sensor530-1, through the image processing unit600of the processor510and stores it in the third block820of the first memory520by each frame unit. The electronic device500performs image processing on an image obtained through the Nth image sensor530-N, through the external image processing unit540-(N−1) and stores it in the first block800of the first memory520by each frame unit. At this point, the processor510can set a time stamp on an image inputted to the image processing unit600or an image outputted from the image processing unit600, and can set a time stamp on an image provided from the external image processing unit540-(N−1). As another example, referring toFIG. 5, the electronic device500performs image processing on an image obtained through the first image sensor530-1, through the image processing unit600of the processor510and stores it in the first memory520by each frame unit. At this point, the processor510can set a time stamp in an image inputted to the image processing unit600or an image outputted from the image processing unit600. The electronic device500can store an image obtained through the Nth image sensor530-N in the second memory550by each frame unit before performing image processing through the image processing control unit700of the external image processing unit540-(N−1). At this point, the external image processing unit540-(N−1) can set a time stamp in an image stored in the second memory550by using the time setting unit710. Here, an image stored in the memory520or550can include metadata having at least one of a frame identifier, a time stamp, focus information, and image setting information EXIF.

The electronic device can select a plurality of images for synthesis on the basis of a time stamp of images stored in a memory in operation903. For example, if a capture event occurs, the electronic device500can select a first image whose time stamp is closest to a capture event occurrence time among images stored in the first block800of the first memory520. Moreover, the electronic device500can select a second image whose time stamp is closest to a capture event occurrence time from among images stored in the third block820of the first memory520. The electronic device500can synthesize a first image selected from the first memory520and a second image as one image. As another example, if a capture event occurs, the electronic device500can select a first image whose time stamp is closest to a capture event occurrence time from among images stored in the first memory520. Additionally, the electronic device500can select a second image whose time stamp is closest to a capture event occurrence time from among images stored in the second memory550. The electronic device500can synthesize a first image selected from the first memory520and a second image as one image.

An electronic device including a plurality of image sensors can have different processing delays on an image obtained through each image sensor by at least one difference in the number of pixels in each image sensor, the sensitivity of an Nth image sensor, the size of an image obtained through an Nth image sensor, or an image processing speed. Accordingly, the electronic device can synchronize images for synthesis as shown inFIG. 10 or 12.

FIG. 10is a flowchart illustrating a method of selecting images for synthesis in an electronic device according to various embodiments of the present disclosure. Below, the flowchart for selecting an image for synthesis is described with reference to a memory configuration shown inFIG. 11. Hereinafter, it is assumed that the electronic device includes the low-pixel first image sensor530-1and the high-pixel Nth image sensor530-N as shown inFIG. 5.

Referring toFIG. 10, the electronic device can set a time stamp on images obtained through a plurality of image sensors in operation1001. For example, referring toFIG. 5, the processor510of the electronic device500can set a time stamp on an image inputted to the image processing unit600or an image outputted from the image processing unit600, and can set a time stamp on an image provided from the external image processing unit540-(N−1). At another example, referring toFIG. 5, the processor510of the electronic device500can set a time stamp on an image inputted to the image processing unit600or an image outputted from the image processing unit600. Additionally, the external image processing unit540-(N−1) of the electronic device500can set a time stamp on an image provided from the Nth image sensor530-N by using the time setting unit710.

The electronic device can display images obtained through image sensors on a display unit in operation1003. At this point, the electronic device can store at least one image displayed on a display unit in at least one memory. For example, when a time stamp is set in an image inputted to the image processing unit600through the processor510, the electronic device500can perform image processing on an image where a time stamp is set through the image processing unit600of the processor510and can store the image in the third block820of the first memory520. As another example, when a time stamp is set in an image outputted from the image processing unit600through the processor510, the electronic device500can store an image whose time stamp is set in the third block820of the first memory520. As another example, when a time stamp is set in an image outputted from the external image processing unit540-(N−1) through the processor510, the electronic device500can store an image whose time stamp is set in the first block800of the first memory520. As another example, when a time stamp is set in an image outputted from the external image processing unit530-N through the time setting unit710, the electronic device500can store an image whose time stamp is set in the second memory550. At this point, referring toFIG. 11, the electronic device can store an image in the memory520or550by a frame unit. Here, an image stored in the memory520or550can include metadata having at least one of a frame identifier, a time stamp, focus information, and image setting information EXIF.

The electronic device can confirm whether a capture event occurs in operation1005. For example, the electronic device can confirm whether a hardware button input corresponding to the capture event is detected. As another example, the electronic device can confirm whether an icon corresponding to the capture event is detected. As another example, the electronic device can confirm whether a user gesture corresponding to the capture event is detected.

If the capture event does not occur, the electronic device can set a time stamp on images obtained through a plurality of image sensors in operation1001.

If the capture event occurs, the electronic device can select at least one image for synthesis including a time stamp prior closest to a capture event occurrence time from among images obtained through a first image sensor stored in a memory in operation1007. For example, referring toFIG. 5, the electronic device500can select one first image1130including a time stamp prior closest to a capture event occurrence time1120as an image for synthesis from among images1100obtained through the first image sensor530-1and sequentially stored in the first memory520. At this point, the electronic device500can select one first image1130from among images including a time stamp prior to the capture event occurrence time1120in consideration of a difference between a processing delay on an image obtained through the first image sensor530-1and a processing delay on an image obtained through the Nth image sensor530-N.

Additionally, if the capture event occurs, the electronic device can select one image for synthesis including a time stamp closest to a capture event occurrence time from among images obtained through the Nth image sensor stored in a memory in operation1009. For example, referring toFIG. 5, the electronic device500can select one second image1140including a time stamp closest to a capture event occurrence time1120as an image for synthesis from among images1110obtained through the Nth image sensor530-N and sequentially stored in the first memory520or the second memory550.

If images are selected for synthesis in operation1007and operation1009, the electronic device can synthesize a first image selected from images provided from a first image sensor with a second image selected from images provided from an Nth image sensor as one image in operation1011.

FIG. 12is a flowchart illustrating a method of selecting images for synthesis in an electronic device according to various embodiments of the present disclosure. Below, the flowchart for selecting an image for synthesis is described with reference to a memory configuration shown inFIG. 13. Hereinafter, it is assumed that the electronic device includes the low-pixel first image sensor530-1and the high-pixel Nth image sensor530-N as shown inFIG. 5.

Referring toFIG. 12, the electronic device can set a time stamp on images obtained through a plurality of image sensors in operation1201. For example, referring toFIG. 5, the processor510of the electronic device500can set a time stamp on an image inputted to the image processing unit600or an image outputted from the image processing unit600. Additionally, the external image processing unit540-(N−1) of the electronic device500can set a time stamp on an image provided from the Nth image sensor530-N by using the time setting unit710.

The electronic device can display images obtained through image sensors on a display unit in operation1203. At this point, the electronic device can store at least one image displayed on a display unit in at least one memory. For example, referring toFIG. 5, when a time stamp is set in an image inputted to the image processing unit600through the processor510, the electronic device500can perform image processing on an image where a time stamp is set through the image processing unit600of the processor510and can store the image in the third block820of the first memory520. As another example, referring toFIG. 5, when a time stamp is set in an image outputted from the image processing unit600through the processor510, the electronic device500can store an image whose time stamp is set in the third block820of the first memory520. As another example, referring toFIG. 7, when a time stamp is set in an image provided from the Nth image sensor530-N through the time setting unit710, the electronic device500can store an image whose time stamp is set in the second memory550. At this point, as shown inFIG. 13, the electronic device can store an image in the memory520or550by a frame unit in operations1300and1310. Here, an image stored in the memory520or550can include metadata having at least one of a frame identifier, a time stamp, focus information, and image setting information EXIF.

The electronic device can confirm whether a capture event occurs in operation1205. For example, the electronic device can confirm whether a hardware button input corresponding to the capture event is detected. As another example, the electronic device can confirm whether an icon corresponding to the capture event is detected. As another example, the electronic device can confirm whether a user gesture corresponding to the capture event is detected.

If the capture event does not occur, the electronic device can set a time stamp on images obtained through a plurality of image sensors in operation1201.

Additionally, if the capture event occurs, the electronic device can select one image for synthesis including a time stamp closest to a capture event occurrence time from among images obtained through the Nth image sensor stored in a memory in operation1207. For example, referring toFIG. 5, the electronic device500can select one second image1330including a time stamp closest to a capture event occurrence time1320as an image for synthesis from among images1310obtained through the Nth image sensor530-N and sequentially stored in the second memory550.

The electronic device can select a second image for synthesis from among images obtained through the Nth image sensor and stored in a memory in order to synthesize one image including a time stamp closest to the processing time of a processor in operation1209. For example, the electronic device500can select one first image1340including a time stamp closest to a time at which the second image selected from the external image processing unit540-(N−1) starts to be transmitted to the processor510in order for synthesis in operation1207. As another example, the electronic device500can select one first image1340including a time stamp closest to a time at which the second image selected from the external image processing unit540-(N−1) starts to be stored in the first memory520through the processor510in order for synthesis in operation1207.

If images are selected for synthesis, the electronic device can synthesize a first image selected from images provided from a first image sensor with a second image selected from images provided from an Nth image sensor as one image in operation1211.

As mentioned above, the electronic device converts at least one image obtained through at least one sensor into a format displayable on a display unit through an image processing unit inside or outside a processor and then provides the converted image to the processor, so that a time delay due to an image processing time can be reduced.

By selecting a plurality of images for synthesis through a time stamp set in images obtained from image sensors, the acquisition times of images to be synthesized can be synchronized.

While the disclosure has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the disclosure as defined by the appended claims. Therefore, the scope of the disclosure is defined not by the detailed description of the disclosure but by the appended claims, and all differences within the scope will be construed as being included in the present disclosure.