Electronic device comprising image sensor for identifying an operation setting and an external environmental condition and method of operation thereof

An electronic device is provided and includes an image sensor including a first unit pixel including a first micro-lens and a plurality of first photodiodes facing each other with a first color filter interposed between the plurality of first photodiodes, and a second unit pixel including a second micro-lens and a plurality of second photodiodes facing each other with a second color filter interposed between the plurality of second photodiodes, a camera module including the image sensor, and a processor operatively connected with the image sensor. The first unit pixel includes a first photodiode, a second photodiode, a third photodiode, and a fourth photodiode, which are disposed in a square shape such that a horizontal number of photodiodes is identical to a vertical number of photodiodes. The second unit pixel includes a fifth photodiode, a sixth photodiode, a seventh photodiode, and an eighth photodiode, which are disposed in a square shape such that a horizontal number of photodiodes is identical to a vertical number of photodiodes. The image sensor identifies an operation setting and an external environmental condition of the camera module and, upon identifying that the operation setting is a preview mode and the external environmental condition is a high luminous environment, identifies a first area signal corresponding to a signal of the first photodiode and the second photodiode, which are concatenated with each other and included in the first unit pixel, and identifies a second area signal that corresponds to a signal of the fifth photodiode, included in the second unit pixel, corresponding to a location of the first photodiode and the sixth photodiode, included in the second unit pixel, corresponding to a location of the second photodiode, and forms first auto focus (AF) information based on the first area signal and the second area signal.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. § 119 to Korean Patent Application No. 10-2019-0172916, filed on Dec. 23, 2019, in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference herein its entirety.

BACKGROUND

The disclosure relates to an electronic device including an image sensor and a method of operating the electronic device.

2. Description of Related Art

An image sensor is a device that converts an optical image into an electrical signal. With the development of computer and communication industries, there is an increasing demand on a high-performance image sensor in various electronic devices such as a digital camera, a camcorder, a personal communication system (PCS), a game console, a security camera, a medical micro camera, and a robot.

The image sensor may include at least one micro-lens. At least one pixel may be disposed under the micro-lens.

In an image sensor including a unit pixel in which a plurality of pixels are disposed under a single micro-lens, it may be difficult to generate an image with a high resolution, using left and right images.

SUMMARY

The present disclosure has been made to address the above-mentioned problems and disadvantages, and to provide at least the advantages described below.

In accordance with an aspect of the disclosure, an electronic device includes an image sensor including a first unit pixel including a first micro-lens and a plurality of first photodiodes facing each other with a first color filter interposed between the plurality of first photodiodes, and a second unit pixel including a second micro-lens and a plurality of second photodiodes facing each other with a second color filter interposed between the plurality of second photodiodes, a camera module including the image sensor, and a processor operatively connected with the image sensor. The first unit pixel includes a first photodiode, a second photodiode, a third photodiode, and a fourth photodiode, which are disposed in a square shape such that a horizontal number of photodiodes is identical to a vertical number of photodiodes. The second unit pixel includes a fifth photodiode, a sixth photodiode, a seventh photodiode, and an eighth photodiode, which are disposed in a square shape such that a horizontal number of photodiodes is identical to a vertical number of photodiodes. The image sensor identifies an operation setting and an external environmental condition of the camera module and, upon identifying that the operation setting is a preview mode and the external environmental condition is a high luminous environment, identifies a first area signal corresponding to a signal of the first photodiode and the second photodiode, which are concatenated with each other and included in the first unit pixel, and identifies a second area signal that corresponds to a signal of the fifth photodiode, included in the second unit pixel, corresponding to a location of the first photodiode and the sixth photodiode, included in the second unit pixel, corresponding to a location of the second photodiode, and forms first auto focus (AF) information based on the first area signal and the second area signal.

In accordance with another aspect of the disclosure, an operating method of an electronic device including an image sensor including a first unit pixel including a first micro-lens and a plurality of photodiodes disposed in a square shape such that a horizontal number of photodiodes is identical to a vertical number of photodiodes, and a second unit pixel including a second micro-lens and a plurality of photodiodes, which face each other and are disposed in a square shape such that a horizontal number of photodiodes is identical to a vertical number of photodiodes, a camera module including the image sensor, and a processor operatively connected with the image sensor identifying, by the image sensor, an operation setting and an external environmental condition of the camera module, and upon identifying that the operation setting is a preview mode and the external environmental condition is a high luminous environment, or upon identifying that the operation setting is a video mode and the external environmental condition is a low luminous environment, identifying a first area signal corresponding to a signal of a first photodiode and a second photodiode, which are concatenated with each other and which are included in the first unit pixel, identifying a second area signal corresponding to a signal of a fifth photodiode corresponding to a location of the first photodiode, and a sixth photodiode corresponding to a location of the second photodiode, wherein the fifth photodiode and the sixth photodiode are included in the second unit pixel, and forming first AF information based on the first area signal and the second area signal.

DETAILED DESCRIPTION

An aspect of the disclosure is to provide an image sensor capable of implementing a high-resolution image.

Hereinafter, various embodiments of the disclosure will be described with reference to accompanying drawings. However, it should be understood that this is not intended to limit the disclosure to specific implementation forms and includes various modifications, equivalents, and/or alternatives of embodiments of the disclosure.

FIG.1is a diagram illustrating an electronic device in a network environment according to various embodiments. Referring toFIG.1, an electronic device101may communicate with an electronic device102through a first network198(e.g., a short-range wireless communication network) or may communicate with an electronic device104or a server108through a second network199(e.g., a long-distance wireless communication network) in a network environment100. According to an embodiment, the electronic device101may communicate with the electronic device104through the server108. According to an embodiment, the electronic device101may include a processor120, a memory130, an input device150, a sound output device155, a display device160, an audio module170, a sensor module176, an interface177, a haptic module179, a camera module180, a power management module188, a battery189, a communication module190, a subscriber identification module196, or an antenna module197. According to some embodiments, at least one among components of the electronic device101may be omitted or one or more other components may be added to the electronic device101. According to some embodiments, some of the above components may be implemented with one integrated circuit. For example, the sensor module176(e.g., a fingerprint sensor, an iris sensor, or an illuminance sensor) may be embedded in the display device160(e.g., a display).

The processor120may execute, for example, software (e.g., a program140) to control at least one of other components (e.g., a hardware or software component) of the electronic device101connected to the processor120and may process or compute a variety of data. According to an embodiment, as a part of data processing or operation, the processor120may load a command set or data, which is received from other components (e.g., the sensor module176or the communication module190), into a volatile memory132, may process the command or data loaded into the volatile memory132, and may store result data into a nonvolatile memory134. According to an embodiment, the processor120may include a main processor121(e.g., a central processing unit or an application processor) and an auxiliary processor123(e.g., a graphic processing device, an image signal processor, a sensor hub processor, or a communication processor), which operates independently from the main processor121or with the main processor121. Additionally or alternatively, the auxiliary processor123may use less power than the main processor121, or is specified to a designated function. The auxiliary processor123may be implemented separately from the main processor121or as a part thereof.

The auxiliary processor123may control, for example, at least some of functions or states associated with at least one component (e.g., the display device160, the sensor module176, or the communication module190) among the components of the electronic device101instead of the main processor121while the main processor121is in an inactive (e.g., sleep) state or together with the main processor121while the main processor121is in an active (e.g., an application execution) state. According to an embodiment, the auxiliary processor123(e.g., the image signal processor or the communication processor) may be implemented as a part of another component (e.g., the camera module180or the communication module190) that is functionally related to the auxiliary processor123.

The memory130may store a variety of data used by at least one component (e.g., the processor120or the sensor module176) of the electronic device101. For example, data may include software (e.g., the program140) and input data or output data with respect to commands associated with the software. The memory130may include the volatile memory132or the nonvolatile memory134.

The program140may be stored in the memory130as software and may include, for example, an operating system142, a middleware144, or an application146.

The input device150may receive a command or data, which is used for a component (e.g., the processor120) of the electronic device101, from an outside (e.g., a user) of the electronic device101. The input device150may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus pen).

The sound output device155may output a sound signal to the outside of the electronic device101. The sound output device155may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia play or recordings play, and the receiver may be used for receiving calls. According to an embodiment, the receiver and the speaker may be either integrally or separately implemented.

The display device160may visually provide information to the outside (e.g., the user) of the electronic device101. For example, the display device160may include a display, a hologram device, or a projector and a control circuit for controlling a corresponding device. According to an embodiment, the display device160may include a touch circuitry configured to sense the touch or a sensor circuit (e.g., a pressure sensor) for measuring an intensity of pressure on the touch.

The audio module170may convert a sound and an electrical signal in dual directions. According to an embodiment, the audio module170may obtain the sound through the input device150or may output the sound through the sound output device155or an external electronic device (e.g., the electronic device102(e.g., a speaker or a headphone)) directly or wirelessly connected to the electronic device101.

The sensor module176may generate an electrical signal or a data value corresponding to an operating state (e.g., power or temperature) inside or an environmental state (e.g., a user state) outside the electronic device101. According to an embodiment, the sensor module176may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface177may support one or more designated protocols to allow the electronic device101to connect directly or wirelessly to the external electronic device (e.g., the electronic device102). According to an embodiment, the interface177may include, for example, an HDMI (high-definition multimedia interface), a USB (universal serial bus) interface, an SD card interface, or an audio interface.

A connecting terminal178may include a connector that physically connects the electronic device101to the external electronic device (e.g., the electronic device102). According to an embodiment, the connecting terminal178may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

The haptic module179may convert an electrical signal to a mechanical stimulation (e.g., vibration or movement) or an electrical stimulation perceived by the user through tactile or kinesthetic sensations. According to an embodiment, the haptic module179may include, for example, a motor, a piezoelectric element, or an electric stimulator.

The camera module180may shoot a still image or a video image. According to an embodiment, the camera module180may include, for example, at least one or more lenses, image sensors, image signal processors, or flashes.

The power management module188may manage power supplied to the electronic device101. According to an embodiment, the power management module188may be implemented as at least a part of a power management integrated circuit (PMIC).

The battery189may supply power to at least one component of the electronic device101. According to an embodiment, the battery189may include, for example, a non-rechargeable (primary) battery, a rechargeable (secondary) battery, or a fuel cell.

The communication module190may establish a direct (e.g., wired) or wireless communication channel between the electronic device101and the external electronic device (e.g., the electronic device102, the electronic device104, or the server108) and support communication execution through the established communication channel. The communication module190may include at least one communication processor operating independently from the processor120(e.g., the application processor) and supporting the direct (e.g., wired) communication or the wireless communication. According to an embodiment, the communication module190may include a wireless communication module192(e.g., a cellular communication module, a short-range wireless communication module, or a GNSS (global navigation satellite system) communication module) or a wired communication module194(e.g., an LAN (local area network) communication module or a power line communication module). The corresponding communication module among the above communication modules may communicate with the external electronic device104through the first network198(e.g., the short-range communication network such as a Bluetooth, a WiFi direct, or an IrDA (infrared data association)) or the second network199(e.g., the long-distance wireless communication network such as a cellular network, an internet, or a computer network (e.g., LAN or WAN)). The above-mentioned various communication modules may be implemented into one component (e.g., a single chip) or into separate components (e.g., chips), respectively. The wireless communication module192may identify and authenticate the electronic device101using user information (e.g., international mobile subscriber identity (IMSI)) stored in the subscriber identification module196in the communication network, such as the first network198or the second network199.

At least some components among the components may be connected to each other through a communication method (e.g., a bus, a GPIO (general purpose input and output), an SPI (serial peripheral interface), or an MIPI (mobile industry processor interface)) used between peripheral devices to exchange signals (e.g., a command or data) with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device101and the external electronic device104through the server108connected to the second network199. Each of the electronic devices102and104may be the same or different types as or from the electronic device101. According to an embodiment, all or some of the operations performed by the electronic device101may be performed by one or more external electronic devices among the external electronic devices102,104, or108. For example, when the electronic device101performs some functions or services automatically or by request from a user or another device, the electronic device101may request one or more external electronic devices to perform at least some of the functions related to the functions or services, in addition to or instead of performing the functions or services by itself. The one or more external electronic devices receiving the request may carry out at least a part of the requested function or service or the additional function or service associated with the request and transmit the execution result to the electronic device101. The electronic device101may provide the result as is or after additional processing as at least a part of the response to the request. To this end, for example, a cloud computing, distributed computing, or client-server computing technology may be used.

FIG.2is a block diagram200illustrating the camera module180according to various embodiments. Referring toFIG.2, the camera module180may include a lens assembly210, a flash220, an image sensor230, an image stabilizer240, a memory250(e.g., buffer memory), or an image signal processor260. The lens assembly210may collect light emitted from an object whose image is to be taken. The lens assembly210may include one or more lenses. According to an embodiment, the camera module180may include a plurality of lens assemblies210. In such a case, the camera module180may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies210may have the same lens attribute (e.g., view angle, focal length, auto-focusing, f number, or optical zoom), or at least one lens assembly may have one or more lens attributes different from those of another lens assembly. The lens assembly210may include, for example, a wide-angle lens or a telephoto lens.

The flash220may emit light that is used to reinforce light emitted or reflected from an object. According to an embodiment, the flash220may include one or more light emitting diodes (LEDs) (e.g., a red-green-blue (RGB) LED, a white LED, an infrared (IR) LED, or an ultraviolet (UV) LED) or a xenon lamp. The image sensor230may obtain an image corresponding to an object by converting light emitted or reflected from the object and transmitted via the lens assembly210into an electrical signal. According to an embodiment, the image sensor230may include one selected from image sensors having different attributes, such as a RGB sensor, a black-and-white (BW) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same attribute, or a plurality of image sensors having different attributes. Each image sensor included in the image sensor230may be implemented using, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

The image stabilizer240may move the image sensor230or at least one lens included in the lens assembly210in a particular direction, or control an operational attribute (e.g., adjust the read-out timing) of the image sensor230in response to the movement of the camera module180or the electronic device101including the camera module180. This allows compensating for at least part of a negative effect (e.g., image blurring) by the movement on an image being captured. According to an embodiment, the image stabilizer240may sense such a movement by the camera module180or the electronic device101using a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module180. According to an embodiment, the image stabilizer240may be implemented, for example, as an optical image stabilizer. The memory250may store, at least temporarily, at least part of an image obtained via the image sensor230for a subsequent image processing task. For example, when the image capturing is delayed due to shutter lag or multiple images are quickly captured, a raw image obtained (e.g., a Bayer-patterned image, a high-resolution image) may be stored in the memory250, and its corresponding copy image (e.g., a low-resolution image) may be previewed via the display device160. Thereafter, when a specified condition is met (e.g., by a user's input or system command), at least part of the raw image stored in the memory250may be obtained and processed, for example, by the image signal processor260. According to an embodiment, the memory250may be configured as at least part of the memory130or as a separate memory that is operated independently from the memory130.

The image signal processor260may perform one or more image processing on an image obtained via the image sensor230or an image stored in the memory250. The one or more image processing may include, for example, depth map generation, three-dimensional (3D) modeling, panorama generation, feature point extraction, image synthesizing, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor260may control (e.g., exposure time control or read-out timing control) at least one (e.g., the image sensor230) of the components included in the camera module180. An image processed by the image signal processor260may be stored back in the memory250for further processing, or may be provided to an external component (e.g., the memory130, the display device160, the electronic device102, the electronic device104, or the server108) outside the camera module180. According to an embodiment, the image signal processor260may be configured as at least part of the processor120, or as a separate processor that is operated independently from the processor120. When the image signal processor260is configured as a separate processor from the processor120, at least one image processed by the image signal processor260may be displayed, by the processor120, via the display device160as it is or after being further processed.

According to an embodiment, the electronic device101may include a plurality of camera modules180having different attributes or functions. In such a case, at least one of the plurality of camera modules180may form, for example, a wide-angle camera and at least another of the plurality of camera modules180may form a telephoto camera. Similarly, at least one of the plurality of camera modules180may form, for example, a front camera and at least another of the plurality of camera modules180may form a rear camera.

Hereinafter, an electronic device will be described with reference toFIGS.3,4, and5. For clarity, contents the same as the above-described details are briefly described or omitted.

FIG.3is a diagram illustrating an image sensor300, according to an embodiment.FIG.4is a diagram400for describing unit pixels, according to an embodiment.FIG.5is a cross-sectional view500illustrating a cross-section of a single unit pixel ofFIG.4taken along line z-z′, according to an embodiment.

Referring toFIG.3, the image sensor300includes a pixel array332, a first scan circuit331, a second scan circuit333, and a read circuit335.

The pixel array332may include a plurality of pixels units including a first pixel unit310, a second pixel unit311, a third pixel unit,312, a fourth pixel unit320, a fifth pixel unit, and a sixth pixel unit322. Additional pixel units may also be included in the plurality of pixel units. The plurality of unit pixels may be arranged along an X direction (e.g., a row direction) and a Y direction (e.g., a column direction).

Each of a plurality of unit pixels may include one micro-lens. For example, the first unit pixel310may include a first micro-lens305and/or the second unit pixel311may include a second micro-lens306.

Each of the plurality of unit pixels may include at least one photodiode (e.g., a first photodiode301, a second photodiode302, a third photodiode303, and a fourth photodiode304). The number of photodiodes included in each of the plurality of unit pixels may be based on the number of pixels included in each of the plurality of unit pixels.

The first scan circuit331and the second scan circuit333may detect signals for each of the plurality of pixels units including the first pixel unit310, the second pixel unit311, the third pixel unit,312, the fourth pixel unit320, the fifth pixel unit, and the sixth pixel unit322under the control of a processor. The first scan circuit331may detect a signal for each of the plurality of pixels units including the first pixel unit310, the second pixel unit311, the third pixel unit,312, the fourth pixel unit320, the fifth pixel unit, and the sixth pixel unit322in the Y direction. The second scan circuit333may detect a signal for each of the plurality of pixels units including the first pixel unit310, the second pixel unit311, the third pixel unit,312, the fourth pixel unit320, the fifth pixel unit, and the sixth pixel unit322in the X direction. The read circuit335may read the detected signal.

Each of the plurality of pixels units including the first pixel unit310, the second pixel unit311, the third pixel unit,312, the fourth pixel unit320, the fifth pixel unit, and the sixth pixel unit322may include n×n pixels (where “n” is a natural number greater than 1). The first unit pixel310may include n×n pixels overlapping with the first micro-lens305and/or the second unit pixel311may include n×n pixels overlapping with the second micro-lens306. Each of the n×n pixels may correspond to a single photodiode.

Each of the plurality of pixels including the first pixel unit310, the second pixel unit311, the third pixel unit,312, the fourth pixel unit320, the fifth pixel unit, and the sixth pixel unit322may include a single micro-lens, a plurality of photodiodes, and a color filter (e.g., green) of one color.

Hereinafter, unit pixels included in the pixel array332of the image sensor300ofFIG.3will be described in detail with reference toFIGS.4and5.

Referring toFIG.4, each of a plurality of unit pixels include 2×2 photodiodes. For example, the first unit pixel310may include the first photodiode301, the second photodiode302, the third photodiode303, and the fourth photodiode304, which are arranged to overlap with the first micro-lens305. The first photodiode301and the third photodiode303may be arranged along the first row. The second photodiode302and the fourth photodiode304may be arranged along the second row. The first photodiode301and the second photodiode302may be arranged along the first column. The third photodiode303and the fourth photodiode304may be arranged along the second column. Each of the first photodiode301, the second photodiode302, the third photodiode303, and the fourth photodiode304may correspond to a single pixel.

The first unit pixel310may include one micro-lens305, the first photodiode301, the second photodiode302, the third photodiode303, the fourth photodiode304, and a green (G) color filter. The second unit pixel311may have the same structure as the first unit pixel310, and thus the description thereof will be omitted. However, the color of the color filter included in each of the unit pixels may be green (G), and may also be different from one another, such as red (R), or blue (B).

Referring toFIG.5, in the first unit pixel310, the first micro-lens305, the first photodiode301and the second photodiodes302may be disposed to face each other with a color filter501interposed therebetween. The light incident on the image sensor may be condensed through the first micro-lens305and may be incident on different photodiodes. The signal for the light incident on each photodiode may be detected by the first scan circuit331or the second scan circuit333.

Hereinafter, the operation of an electronic device according to an embodiment will be described with reference toFIGS.6,7,8, and9. For clarity, contents which are the same as the above-described details are briefly described or omitted.

Hereinafter, it is assumed that the image sensor300ofFIG.3performs the process ofFIG.6. The operation described as being performed by the image sensor300may be performed by a synthetic signal forming unit701, a disparity calculation unit702, and/or a remosaic processing unit703included in the image sensor300.

FIG.6is a flowchart600for describing an operation of an image sensor300, according to an embodiment.FIG.7is a block diagram700illustrating pieces of hardware included in an image sensor300, according to an embodiment.FIG.8is a diagram800illustrating unit pixels included in an image sensor300, according to an embodiment.FIG.9is a diagram900illustrating an example of forming a Bayer-patterned image based on signals of an image sensor, according to an embodiment.

Referring toFIG.6, in step610, an image sensor identifies camera operation settings and/or external environmental conditions.

The camera operation settings may indicate which of a preview mode, a photo mode, and a video mode the mode of a camera module180included in an electronic device101corresponds to. The preview mode may be a mode in which a screen captured by the image sensor is provided as a preview to a user when the camera module does not perform photo shooting or video shooting. The photo mode may be a mode in which the camera module takes a photo. The video mode may be a mode in which the camera module captures a video.

The external environmental conditions may indicate to which of high illuminance and low illuminance the external environment of the electronic device corresponds. The high illuminance may indicate that a brightness value (BV) corresponding to lux is not less than a specific threshold value. The low illuminance may indicate that the BV is less than the specific threshold value.

The image sensor may not directly identify camera operation settings and external environmental conditions, but may receive the information identified by a processor120from the processor of the electronic device.

In step620, the image sensor determines the operation of the image sensor and the signal to be output, based on the camera operation settings and/or external environmental conditions identified in step610. Hereinafter, the operation and output signals of the image sensor according to the identified camera operation settings and/or external environmental conditions will be described in detail with reference to Table 1, below.

In Table 1, the first camera operation setting, the second camera operation setting, and the third camera operation setting may be different modes of the camera module180, respectively. For example, the first camera operation setting may be in a preview mode; the second camera operation setting may be in a video mode; the third camera operation setting may be in a photo mode.

In Table 1, the first external environmental condition and the second external environmental condition may indicate different conditions for the surrounding environment of the camera module180. For example, the first external environmental condition may be a high illuminance condition; the second external environmental condition may be a low illuminance condition.

In Table 1, the operation of the image sensor may indicate which signal of the image sensor is read out by the image sensor, depending on the identified camera operation settings and/or external environmental conditions.

In Table 1, AF information may be information about auto focus, and may mean what AF information the image sensor forms and outputs to the processor120, depending on the identified camera operation settings and/or external environmental conditions.

In Table 1, the output signal may be a signal to be output by the display device160of the electronic device, and may mean which output signal the image sensor forms and outputs to the processor120, depending on the identified camera operation settings and/or external environmental conditions.

Referring to Table 1, the image sensor may detect a first vertical area signal, a second vertical area signal, a third vertical area signal, and a fourth vertical area signal, and may generate and output first AF information, second AF information, and a first output signal, based on the first camera operation setting and the first external environmental condition. Hereinafter, the vertical area signal, AF information, and output signal will be described in detail with reference toFIGS.7and8.

Referring toFIGS.7and8, the image sensor300includes the synthetic signal forming unit701, the disparity calculation unit702, and the remosaic processing unit703. The synthetic signal forming unit701, the disparity calculation unit702, and the remosaic processing unit703ofFIG.7are illustrated as a block diagram of pieces of hardware included in the image sensor300. However, the synthetic signal forming unit701, the disparity calculation unit702, and the remosaic processing unit703may be implemented with software.

The image sensor300may detect the first vertical area signal, the second vertical area signal, the third vertical area signal, and the fourth vertical area signal, based on the first camera operation setting and the first external environmental condition

The image sensor300may detect a first vertical area signal816, which is a signal for light incident on a first photodiode811and a second photodiode812through a first micro-lens815of a first unit pixel810, through a scan circuit and may transmit the first vertical area signal816to the synthetic signal forming unit701of the image sensor300, based on the first camera operation setting and the first external environmental condition.

In the same manner, the image sensor300may transmit a third vertical area signal826, which is a signal for light incident on a fifth photodiode821and a sixth photodiode822through a second micro-lens825of a second unit pixel820, to the synthetic signal forming unit701based on the first camera operation setting mode and the first external environmental condition.

In the same manner, the image sensor300may transmit a second vertical area signal817, which is a signal for light incident on a third photodiode813and a fourth photodiode814, to the synthetic signal forming unit701based on the first camera operation setting mode and the first external environmental condition.

In the same manner, the image sensor300may transmit a fourth vertical area signal827, which is a signal for light incident on a seventh photodiode823and an eighth photodiode824, to the synthetic signal forming unit701based on the first camera operation setting mode and the first external environmental condition.

In this way, the reason that the image sensor300does not detect the signal of each of a first photodiode811, a second photodiode812, a third photodiode, a fourth photodiode, a fifth photodiode, and a sixth photodiode, but detects the signal by grouping the photodiodes for each area, may be because the power consumption of an electronic device may be excessive, when a screen is output in real time with a high resolution in a high luminous environment because a preview mode needs to output the screen in real time. Accordingly, the signal to be detected by the image sensor may be reduced to ½ by detecting a signal by grouping photodiodes for each area, and thus this may be to reduce the power consumption.

The image sensor300may form first AF information and second AF information and may output the first AF information and the second AF information to a processor120, based on the first camera operation setting and the first external environmental condition.

The synthetic signal forming unit701of the image sensor300may synthesize the received first vertical area signal816and the received third vertical area signal826to form the first AF information.

The synthetic signal forming unit701of the image sensor300may synthesize the received second vertical area signal817and the received fourth vertical area signal827to form the second AF information.

The image sensor300may output the formed first AF information and the formed second AF information to a processor120.

The fact that the image sensor300may only output the first AF information and the second AF information, which are horizontal focus information, to the processor120may be to reduce the power consumption of the electronic device by adjusting the focus only with horizontal focus information without vertical focus information because it is easy to adjust the focus in a high luminous environment. Accordingly, the image sensor outputting horizontal focus information to the processor is described. However, the image sensor may also output only vertical focus information to the processor. The vertical focus information will be described later.

The image sensor300may form a first output signal to output the first output signal to a processor120, based on the first camera operation setting and the first external environmental condition. The image sensor300may generate the first output signal that is a sum of the first vertical area signal816and the second vertical area signal817of the first unit pixel810, to output the first output signal to the processor, based on the first camera operation setting and the first external environmental condition. The image sensor300may generate an output signal corresponding to the first output signal of each of the unit pixels included in a pixel array332to output the output signal to the processor, based on the first camera operation setting and the first external environmental condition.

The fact that the image sensor300may output a plurality of photodiodes included in a unit pixel as a single signal without processing the signal of each of the plurality of photodiodes included in the unit pixel may be to reduce the power consumption of the electronic device even though a resolution decreases because the resolution has low importance in a preview mode.

Returning to Table 1, the image sensor300may detect the first pixel signal, the second pixel signal, the third pixel signal, and the fourth pixel signal per the first unit pixel810; may detect the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal per the second unit pixel820; and may generate and output the first AF information, the second AF information, the third AF information, the fourth AF information, and the second output signal, based on the first camera operation setting and the second external environmental condition. Hereinafter, this will be described in detail with reference toFIGS.7and9.

Referring toFIGS.7and9, the image sensor300may detect the first pixel signal, the second pixel signal, the third pixel signal, and the fourth pixel signal per the first unit pixel810based on the first camera operation setting and the second external environmental condition. The first pixel signal may correspond to a signal for light incident on the first photodiode811. The second pixel signal may correspond to a signal for light incident on the second photodiode812. The third pixel signal may correspond to a signal for light incident on the third photodiode813. The fourth pixel signal may correspond to a signal for light incident on the fourth photodiode814.

The image sensor300may detect the detected first pixel signal, the detected second pixel signal, the detected third pixel signal, and the detected fourth pixel signal through a scan circuit to transmit the detected first pixel signal, the detected second pixel signal, the detected third pixel signal, and the detected fourth pixel signal to the synthetic signal forming unit701of the image sensor300.

The image sensor300may detect the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal per the second unit pixel820based on the first camera operation setting and the second external environmental condition. The fifth pixel signal may correspond to a signal for light incident on the fifth photodiode821. The sixth pixel signal may correspond to a signal for light incident on the sixth photodiode822. The seventh pixel signal may correspond to a signal for light incident on the seventh photodiode823. The eighth pixel signal may correspond to a signal for light incident on the eighth photodiode824.

The image sensor300may detect the detected fifth pixel signal, the detected sixth pixel signal, the detected seventh pixel signal, and the detected eighth pixel signal through a scan circuit to transmit the detected fifth pixel signal, the detected sixth pixel signal, the detected seventh pixel signal, and the detected eighth pixel signal to the synthetic signal forming unit701of the image sensor300.

The image sensor300may form the first AF information, the second AF information, the third AF information, and the fourth AF information to output the first AF information, the second AF information, the third AF information, and the fourth AF information to a processor120, based on the first camera operation setting and the second external environmental condition.

The synthetic signal forming unit701of the image sensor300may synthesize the received first pixel signal, the received second pixel signal, the received fifth pixel signal, and the received sixth pixel signal to form the first AF information.

The synthetic signal forming unit701of the image sensor300may synthesize the received third pixel signal, the received fourth pixel signal, the received seventh pixel signal, and the received eighth pixel signal to form the second AF information.

The synthetic signal forming unit701of the image sensor300may synthesize the received first pixel signal, the received third pixel signal, the received fifth pixel signal, and the received seventh pixel signal to form the third AF information.

The synthetic signal forming unit701of the image sensor300may synthesize the received second pixel signal, the received fourth pixel signal, the received sixth pixel signal, and the received eighth pixel signal to form the fourth AF information.

The image sensor300may output the formed first AF information, the formed second AF information, the formed third AF information, and the formed fourth AF information to the processor120.

The fact that the image sensor300may form all of the first AF information and the second AF information that are horizontal focus information, and the third AF information and the fourth AF information that are vertical focus information to output the first AF information, the second AF information, the third AF information, and the fourth AF information to the processor120may be to complement the vertical focus information and the horizontal focus information with each other to adjust a focus by using both the vertical focus information and the horizontal focus information because it may be difficult to adjust the focus in a low luminous environment.

The image sensor300may form a second output signal to output the second output signal to the processor120, based on the first camera operation setting and the second external environmental condition. The image sensor300may generate the second output signal obtained by summing the first pixel signal, the second pixel signal, the third pixel signal, and the fourth pixel signal to output the summed result to the processor based on the first camera operation setting and the second external environmental condition. The image sensor300may generate the output signal corresponding to the second output signal of each of the unit pixels included in a pixel array332based on the first camera operation setting and the second external environmental condition.

The reason that the image sensor300sums signals of a plurality of photodiodes included in a unit pixel to output the summed result as a single signal (the second output signal) may be because the resolution of the displayed screen has low importance in the preview mode.

Returning to Table 1, the image sensor300may detect the first pixel signal, the second pixel signal, the third pixel signal, and the fourth pixel signal per the first unit pixel810; may detect the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal per the second unit pixel820; and may generate and output the first AF information, the second AF information, the third AF information, the fourth AF information, and the third output signal, based on the second camera operation setting and the first external environmental condition. The description about the operation and AF information output of the image sensor300based on the second camera operation setting and the first external environmental condition may be the same as that in the case of the first camera operation setting and the second external environmental condition, and thus the description will be omitted. The third output signal may correspond to the Bayer-patterned image. Hereinafter, the third output signal will be described in detail with reference toFIGS.7and9.

Referring toFIGS.7and9, a pixel array910of the image sensor300may include a plurality of photodiodes including a first photodiode811, a second photodiode812, a third photodiode813, a fourth photodiode814, a fifth photodiode821, a sixth photodiode822, a seventh photodiode823, and an eighth photodiode824. Additional photodiodes may also be included in the plurality of photodiodes. The image sensor300may detect the first pixel signal, the second pixel signal, the third pixel signal, the fourth pixel signal, the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal from the plurality of photodiodes including the first photodiode811, the second photodiode812, the third photodiode813, the fourth photodiode814, the fifth photodiode821, the sixth photodiode822, the seventh photodiode823, and the eighth photodiode824based on the second camera operation setting and the first external environmental condition, respectively.

The remosaic processing unit703of the image sensor300may form a Bayer-patterned image920by converting the detected first pixel signal, the detected second pixel signal, the detected third pixel signal, the detected fourth pixel signal, the detected fifth pixel signal, the detected sixth pixel signal, the detected seventh pixel signal, and the detected eighth pixel signal.

The Bayer-patterned image may mean a pattern in which green (G) and each of red (R) and blue (B) are intersected with each other such that green (G) is 50%, and each of red (R) and blue (B) are 25% depending on visual characteristics of a human. Additionally, it may be necessary to convert signals detected by the image sensor into Bayer-patterned images when the image signal processor260performs image signal processing (ISP).

A process of converting signals (the first pixel signal, the second pixel signal, the third pixel signal, the fourth pixel signal, the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal) detected by the image sensor into the Bayer-patterned image920may include a process such as rearranging, synthesizing, averaging, weighting, and adjusting contrast of an edge portion of the signals detected by the image sensor300. The algorithm applied to the signals detected by the image sensor300to form the Bayer-patterned image920may be referred to as a remosaic algorithm. The content of the remosaic algorithm may vary for each manufacturer of an image sensor. The algorithm applied to the signals detected by the image sensor to form an image may be collectively referred to as a remosaic algorithm.

The image sensor300may form a third output signal, which is the Bayer-patterned image920, to output the third output signal to a processor120, based on the second camera operation setting and the first external environmental condition.

The reason that the image sensor300may detect the signal of each of a plurality of photodiodes included in a unit pixel to form and output a Bayer-patterned image through processing may be because that maintaining a resolution may be more important than reducing a power consumption of an electronic device, in a video mode. Returning to Table 1, the image sensor300may detect a first vertical area signal, a second vertical area signal, a third vertical area signal, and a fourth vertical area signal, and may generate and output first AF information, second AF information, and a first output signal, based on the second camera operation setting and the second external environmental condition. The description about the operation, AF information, and output signal of the image sensor300based on the second camera operation setting and the second external environmental condition may be the same as that in the case of the first camera operation setting and the first external environmental condition, and thus the description will be omitted.

The fact that the image sensor300may detect the signal by grouping a plurality of photodiodes by area without detecting the signal of each of the plurality of photodiodes included in the unit pixel and output only the first AF information and the second AF information, which are horizontal focus information, to a processor when the image sensor300is in the second camera operation setting and the second external environmental condition may be to reduce detection noise (read out noise) by reducing the signal detected by the image sensor to complement the quality of a video in low luminous environments because sufficient exposure time is not secured in low illuminance.

Returning to Table 1, the image sensor300may detect the first pixel signal, the second pixel signal, the third pixel signal, and the fourth pixel signal per the first unit pixel810; may detect the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal per the second unit pixel820; and may generate and output the third output signal, based on the third camera operation setting and the first external environmental condition. The description about the operation and output signal of the image sensor300based on the third camera operation setting and the first external environmental condition may be the same as that in the case of the second camera operation setting and the first external environmental condition, and thus the description will be omitted.

The image sensor300may not generate AF information in the case of the third camera operation setting and the first external environmental condition. This may be because there is no need to separately form AF information in a photo mode as long as the AF information formed in the preview mode is used. Furthermore, the fact that the image sensor300forms the third output signal (a Bayer-patterned image) when the image sensor300is in the third camera operation setting and the first external environmental condition may be to output a high-resolution photo in a high luminous environment. Returning to Table 1, the image sensor300may detect the first unit pixel signal and the second unit pixel signal and may output the first unit pixel signal and the second unit pixel signal, based on the third camera operation setting and the second external environmental condition.

When the image sensor300is in the third camera operation setting and the second external environmental condition, the image sensor300may detect the first unit pixel signal that is a signal corresponding to the first unit pixel810. Besides, the image sensor300may detect the second unit pixel signal that is a signal corresponding to the second unit pixel820. The image sensor300may output the detected first unit pixel signal and the detected second unit pixel signal to a processor. The reason that the image sensor300groups and detects signals of a plurality of photodiodes included in a unit pixel in the third camera operation setting and the second external environmental condition may be focused on reducing the power consumption of an electronic device because the resolution of photo may be meaningless in a low luminous environment.

The image sensor300may not generate AF information in the case of the third camera operation setting and the second external environmental condition. This may be because there is no need to separately form AF information in a photo mode as long as the AF information formed in the preview mode is used.

Hereinafter, the output signals of an image sensor will be described with reference toFIGS.10and11. The same configurations as those of the above-described embodiment may be referenced by the same reference numerals, and description thereof may be omitted.

FIG.10is a diagram illustrating a first case1010and a second case1040of outputting signals of an image sensor, according to an embodiment.FIG.11is a diagram1100illustrating a method of calculating a disparity of an image sensor, according to an embodiment.

Referring toFIG.10, the image sensor300outputs first output signals1011or second output signals1021to a processor120. The case where the image sensor300outputs the first output signals1011will be described first. The first output signals1011may be signals for outputting a single frame. In a case of the first camera operation setting and the second external environmental condition or in a case of the second camera operation setting and the first external environmental condition, the first output signals1011may be signals that allow the image sensor300to output a single frame.

The image sensor300may not transmit all pieces of horizontal focus information (horizontal Y) including partial signal1013, partial signal1016, and partial signal1019, and all pieces of vertical focus information (vertical Y) including partial signal1014, partial signal1017, and partial signal1020after transmitting all image signals including partial signal1012, partial signal1015, and partial signal1018for a single frame, but alternately transmit signals, for example, a partial signal1012of image signals, a partial signal1013of horizontal focus information, a partial signal1014of vertical focus information, a partial signal1015of image signals, a partial signal1016of horizontal focus information, and a partial signal1017of vertical focus information. As described above, the horizontal focus information may include the first AF information and second AF information. In addition, the vertical focus information may include the third AF information and the fourth AF information.

Each of the signals1012to1020for a single frame may be divided into a virtual channel (VC) and data type (DT) of separate signal specification (e.g., mobile industry processor interface (MIPI)) and may be transmitted to the processor. Examples of VC and DT for each signal are shown in Table 2, below

In Table 2, vertical disparity information (Vertical disparity) will be described later.

Hereinafter, the case where the image sensor300outputs the second output signals1021is described. The second output signals1021may be signals for outputting a single frame. In the case of the first camera operation setting and the second external environmental condition or in the case of the second camera operation setting and the first external environmental condition, the second output signals1021may be signals that allow the image sensor300to output a single frame.

The image sensor300may alternately transmit signals, for example, a partial signal1022of image signals, a partial signal1023of horizontal focus information, a partial signal1024of vertical disparity information (Vertical disparity), a partial signal1025of image signals, a partial signal1026of horizontal focus information, and a partial signal1027of vertical disparity information.

Hereinafter, disparity information will be described in detail with reference toFIGS.10and11. The same configurations as those of the above-described embodiment may be referenced by the same reference numerals, and description thereof may be omitted.

The image sensor300may form a left signal image1120by synthesizing the first vertical area signal816and the third vertical area signal826. That is, the image sensor300may form the left signal image1120based on the first AF information.

Moreover, the image sensor230may form a right signal image1121by synthesizing the second vertical area signal817and the fourth vertical area signal827. That is, the image sensor300may form the right signal image1121based on the second AF information.

The disparity calculation unit702of the image sensor230may calculate horizontal disparity information (Horizontal disparity), which is the disparity between the left signal image including partial signal1013, partial signal1016, or partial signal1019and the right signal image including partial signal1013, partial signal1016, or partial signal1019. The horizontal disparity may be a distance between the center line c1of the left signal image including partial signal1013, partial signal1016, or partial signal1019and the center line c2of the right signal image including partial signal1013, partial signal1016, or partial signal1019. As the horizontal disparity is greater, this may mean that the extent to which the image captured by the image sensor300is out of focus is greater. When the center line c1of the left signal image including partial signal1013, partial signal1016, or partial signal1019completely overlaps with the center line c2of the right signal image including partial signal1013, partial signal1016, or partial signal1019, the horizontal disparity1130is 0. This may mean that the image sensor300is in the on-focus state. Identically, the disparity calculation unit702of the image sensor230may calculate vertical disparity information (Vertical disparity) including partial signal1024, partial signal1027, or partial signal1030that is the disparity between the upper signal image including partial signal1014, partial signal1017, or partial signal1020and the lower signal image including partial signal1014, partial signal1017, or partial signal1020.

Returning toFIG.10, the image sensor300may output vertical disparity information (Vertical disparity) including partial signal1024, partial signal1027, or partial signal1030to a processor instead of vertical focus information (Vertical Y) including partial signal1014, partial signal1017, or partial signal1020.

Hereinafter, an embodiment in which a unit pixel included in a pixel array of an image sensor includes a 3×3 array of photodiodes will be described in detail with reference toFIG.12.

FIG.12is a diagram1200illustrating unit pixels included in a pixel array of an image sensor, according to an embodiment. The operation of the image sensor is the same as in the above-described embodiment, and thus only the definition of the vertical area signal and AF information in the case where a unit pixel includes a 3×3 array of photodiodes will be described.

Referring toFIG.12, each of a plurality of unit pixels include 3×3 photodiodes. For example, a first unit pixel1210includes a first photodiode1211, a second photodiode1212, a third photodiode1213, a fourth photodiode1214, a fifth photodiode1215, a sixth photodiode1216, a seventh photodiode1217, an eight photodiode1218, and a ninth photodiode1219arranged to overlap with a first micro-lens1231. The description of the first unit pixel1210may also be applied to a second unit pixel1220.

The first vertical area signal of the first unit pixel1210may mean a signal detected in the first photodiode1211, the second photodiode1212, the third photodiode1213, the fourth photodiode1214, the fifth photodiode1215, and the sixth photodiode1216. The second vertical area of the first unit pixel1210may include the fourth photodiode1214, the fifth photodiode1215, the sixth photodiode1216, the seventh photodiode1217, the eighth photodiode1218, and the ninth photodiode1219.

The third vertical area of the second unit pixel1220may include a tenth photodiode1221, an eleventh photodiode1222, a twelfth photodiode1223, the thirteenth photodiode1224, the fourteenth photodiode1225, and the fifteenth photodiode1226. The fourth vertical area of the second unit pixel1220may include the thirteenth photodiode1224, the fourteenth photodiode1225, the fifteenth photodiode1226, a sixteenth photodiode1227, a seventeenth photodiode1228, and an eighteenth photodiode1229.

The first AF information may be formed based on the first vertical area signal of the first unit pixel1210and the third vertical area signal of the second unit pixel1220. The second AF information may be formed based on the second vertical area signal of the first unit pixel1210and the fourth vertical area signal of the second unit pixel1220.

The unit pixel is not limited to 2×2 photodiodes or 3×3 photodiodes. For example, an image sensor may include n×n photodiodes per unit pixel.

According to an embodiment of the disclosure, an electronic device may include an image sensor including a first unit pixel including a first micro-lens and a plurality of first photodiodes facing each other with a first color filter interposed between the plurality of first photodiodes, and a second unit pixel including a second micro-lens and a plurality of second photodiodes facing each other with a second color filter interposed between the plurality of second photodiodes, a camera module including the image sensor, and a processor operatively connected with the image sensor. The first unit pixel may include a first photodiode, a second photodiode, a third photodiode, and a fourth photodiode, which are disposed in a square shape such that a horizontal number of photodiodes is identical to a vertical number of photodiodes. The second unit pixel may include a fifth photodiode, a sixth photodiode, a seventh photodiode, and an eighth photodiode, which are disposed in a square shape such that a horizontal number of photodiodes is identical to a vertical number of photodiodes. The image sensor may identify an operation setting and an external environmental condition of the camera module, and, upon identifying that the operation setting is a preview mode and the external environmental condition is a high luminous environment, the image sensor may identify a first area signal corresponding to a signal of the first photodiode and the second photodiode, which are concatenated with each other and included in the first unit pixel, and the image sensor may identify a second area signal that corresponds to a signal of the fifth photodiode, included in the second unit pixel, corresponding to a location of the first photodiode and the sixth photodiode, included in the second unit pixel corresponding to a location of the second photodiode, and may form first AF information based on the first area signal and the second area signal.

Upon identifying that the operation setting is the preview mode and the external environmental condition is the high luminous environment, the image sensor may identify a third area signal corresponding to a signal of the third photodiode and the fourth photodiode, which are concatenated with each other and included in the first unit pixel, may identify a fourth area signal that corresponds to a signal of the seventh photodiode and the eighth photodiode in the second unit pixel, may form second AF information based on the third area signal and the fourth area signal, and may output the formed first AF information and the formed second AF information to the processor.

The image sensor may output an image signal for a preview image to the processor, and the image signal may include a signal obtained by synthesizing the first area signal and the second area signal, and a signal obtained by synthesizing the third area signal and the fourth area signal.

Upon identifying that the operation setting is the preview mode and the external environmental condition is a low luminous environment, the image sensor may form the first AF information based on a first pixel signal of the first photodiode, a second pixel signal of the second photodiode, a fifth pixel signal of the fifth photodiode, and a sixth pixel signal of the sixth photodiode; may form second AF information based on a third pixel signal of the third photodiode, a fourth pixel signal of the fourth photodiode, a seventh pixel signal of the seventh photodiode, and an eighth pixel signal of the eighth photodiode; may form third AF information based on the first pixel signal of the first photodiode, the third pixel signal of the third photodiode, the fifth pixel signal of the fifth photodiode, and the seventh pixel signal of the seventh photodiode; may form fourth AF information based on the second pixel signal of the second photodiode, the fourth pixel signal of the fourth photodiode, the sixth pixel signal of the sixth photodiode, and the eighth pixel signal of the eighth photodiode; may output the formed first AF information and the formed second AF information to the processor; and may output the formed third AF information and the formed fourth AF information to the processor.

The image sensor outputs an image signal for a preview image to the processor, and the image signal may include a signal obtained by synthesizing the first pixel signal, the second pixel signal, the third pixel signal, and the fourth pixel signal, and a signal obtained by synthesizing the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal.

Upon identifying that the operation setting is a video mode and the external environmental condition is the high luminous environment, the image sensor may identify a first pixel signal of the first photodiode, a second pixel signal of the second photodiode, a third pixel signal of the third photodiode, a fourth pixel signal of the fourth photodiode, a fifth pixel signal of the fifth photodiode, a sixth pixel signal of the sixth photodiode, a seventh pixel signal of the seventh photodiode, and an eighth pixel signal of the eighth photodiode; may form a Bayer-patterned image by applying a remosaic algorithm to the first pixel signal, the second pixel signal, the third pixel signal, the fourth pixel signal, the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal; and may output the Bayer-patterned image as an image signal to the processor.

The image sensor may form the first AF information based on the first pixel signal, the second pixel signal, the fifth pixel signal and the sixth pixel signal; may form second AF information based on the third pixel signal, the fourth pixel signal, the seventh pixel signal and the eighth pixel signal; may form third AF information based on the first pixel signal, the third pixel signal, the fifth pixel signal and the seventh pixel signal; may form fourth AF information based on the second pixel signal, the fourth pixel signal, the sixth pixel signal and the eighth pixel signal; may output the formed first AF information and the formed second AF information to the processor; and may output the formed third AF information and the formed fourth AF information to the processor.

Upon identifying that the operation setting is a video mode and the external environmental condition is a low luminous environment, the image sensor may identify the first area signal corresponding to the signal of the first photodiode and the second photodiode, may identify the second area signal corresponding to the signal of the fifth photodiode and the sixth photodiode, may form the first AF information based on the first area signal and the second area signal, may identify a third area signal corresponding to a signal of the third photodiode and the fourth photodiode, may identify a fourth area signal that corresponds to a signal of the seventh photodiode and the eighth photodiode, may form second AF information based on the third area signal and the fourth area signal, and may output the formed first AF information and the formed second AF information to the processor.

The image sensor may output an image signal for a video captured with the camera module, to the processor, and the image signal may include a signal obtained by synthesizing the first area signal and the second area signal, and a signal obtained by synthesizing the third area signal and the fourth area signal.

Upon identifying that the operation setting is a photo mode and the external environmental condition is a high luminous environment, the image sensor may identify a first pixel signal of the first photodiode, a second pixel signal of the second photodiode, a third pixel signal of the third photodiode, a fourth pixel signal of the fourth photodiode, a fifth pixel signal of the fifth photodiode, a sixth pixel signal of the sixth photodiode, a seventh pixel signal of the seventh photodiode, and an eighth pixel signal of the eighth photodiode; may form a Bayer-patterned image by applying a remosaic algorithm to the first pixel signal, the second pixel signal, the third pixel signal, the fourth pixel signal, the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal; and may output the Bayer-patterned image as an image signal to the processor.

Upon identifying the operation settings is a photo mode and the external environmental condition is a low luminous environment, the image sensor may identify a first unit pixel signal that is a signal of the first photodiode, the second photodiode, the third photodiode and the fourth photodiode; may identify a second unit pixel signal that is a signal of the fifth photodiode, the sixth photodiode, the seventh photodiode and the eighth photodiode; and may output an image signal including the first unit pixel signal and the second unit pixel signal to the processor.

According to an embodiment of the disclosure, an operating method of an electronic device including an image sensor including a first unit pixel including a first micro-lens and a plurality of photodiodes disposed in a square shape such that a horizontal number of photodiodes is identical to a vertical number of photodiodes, and a second unit pixel including a second micro-lens and a plurality of photodiodes, which face each other and are disposed in a square shape such that a horizontal number of photodiodes is identical to a vertical number of photodiodes, a camera module including the image sensor, and a processor operatively connected with the image sensor may include identifying, by the image sensor, an operation setting and an external environmental condition of the camera module, upon identifying that the operation setting is a preview mode and the external environmental condition is a high luminous environment, or upon identifying that the operation setting is a video mode and the external environmental condition is a low luminous environment, identifying a first area signal corresponding to a signal of a first photodiode and a second photodiode, which are concatenated with each other and which are included in the first unit pixel, identifying a second area signal corresponding to a signal of a fifth photodiode corresponding to a location of the first photodiode, and a sixth photodiode corresponding to a location of the second photodiode, wherein the fifth photodiode and the sixth photodiode are included in the second unit pixel, and forming first AF information based on the first area signal and the second area signal.

Upon identifying that the operation setting is the preview mode and the external environmental condition is the high luminous environment, or upon identifying that the operation setting is the video mode and the external environmental condition is the low luminous environment, the method may further include identifying, by the image sensor, a third area signal corresponding to a signal of a third photodiode and a fourth photodiode, which are concatenated with each other and which are included in the first unit pixel, identifying, by the image sensor, a fourth area signal corresponding to a signal of a seventh photodiode and an eighth photodiode, which are included in the second unit pixel, forming, by the image sensor, second AF information based on the third area signal and the fourth area signal, and outputting, by the image sensor, the formed first AF information and the formed second AF information to the processor.

The method may further include outputting, by the image sensor, an image signal for a preview image to the processor. The image signal may include a signal obtained by synthesizing the first area signal and the second area signal, and a signal obtained by synthesizing a third area signal and a fourth area signal.

Upon identifying that the operation setting is the preview mode and the external environmental condition is the low luminous environment, or upon identifying that the operation setting is the video mode and the external environmental condition is the high luminous environment, the method may further include forming the first AF information based on a first pixel signal of the first photodiode, a second pixel signal of the second photodiode, a fifth pixel signal of the fifth photodiode, and a sixth pixel signal of the sixth photodiode; forming second AF information based on a third pixel signal of a third photodiode, a fourth pixel signal of a fourth photodiode, a seventh pixel signal of a seventh photodiode, and an eighth pixel signal of an eighth photodiode; forming third AF information based on the first pixel signal of the first photodiode, the third pixel signal of the third photodiode, the fifth pixel signal of the fifth photodiode, and the seventh pixel signal of the seventh photodiode; forming fourth AF information based on the second pixel signal of the second photodiode, the fourth pixel signal of the fourth photodiode, the sixth pixel signal of the sixth photodiode, and the eighth pixel signal of the eighth photodiode; outputting the formed first AF information and the formed second AF information to the processor; and outputting the formed third AF information and the formed fourth AF information to the processor.

Upon identifying that the operation setting is the preview mode and the external environmental condition is the low luminous environment, the image sensor may output an image signal for a preview image to the processor. The image signal may include a signal obtained by synthesizing the first pixel signal, the second pixel signal, the third pixel signal, and the fourth pixel signal, and a signal obtained by synthesizing the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal.

Upon identifying that the operation setting is the video mode and the external environmental condition is the high luminous environment, the method may further include forming a Bayer-patterned image by applying a remosaic algorithm to the first pixel signal, the second pixel signal, the third pixel signal, the fourth pixel signal, the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal, and outputting the Bayer-patterned image as an image signal to the processor.

Upon identifying that the operation setting is a photo mode and the external environmental condition is the high luminous environment, the method may further include identifying a first pixel signal of the first photodiode, a second pixel signal of the second photodiode, a third pixel signal of a third photodiode, a fourth pixel signal of a fourth photodiode, a fifth pixel signal of the fifth photodiode, a sixth pixel signal of the sixth photodiode, a seventh pixel signal of a seventh photodiode, and an eighth pixel signal of an eighth photodiode; forming a Bayer-patterned image by applying a remosaic algorithm to the first pixel signal, the second pixel signal, the third pixel signal, the fourth pixel signal, the fifth pixel signal, the sixth pixel signal, the seventh pixel signal, and the eighth pixel signal; and outputting the Bayer-patterned image as an image signal to the processor.

Upon identifying that the operation setting is a photo mode and the external environmental condition is the low luminous environment, the method may further include identifying a first unit pixel signal that is a signal of the first photodiode, the second photodiode, a third photodiode and a fourth photodiode; identifying a second unit pixel signal that is a signal of the fifth photodiode, the sixth photodiode, a seventh photodiode and an eighth photodiode; and outputting an image signal including the first unit pixel signal and the second unit pixel signal to the processor.

It should be understood that various embodiments of the disclosure and terms used in the embodiments do not intend to limit technical features to the particular embodiment disclosed herein; rather, the disclosure should be construed to cover various modifications, equivalents, or alternatives of embodiments of the disclosure. With regard to description of drawings, similar or related components may be assigned with similar reference numerals. As used herein, singular forms of words corresponding to an item may include one or more items unless the context clearly indicates otherwise. In the disclosure, each of the expressions “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B, or C”, “one or more of A, B, and C”, or “one or more of A, B, or C”, may include any and all combinations of one or more of the associated listed items. The expressions, such as “a first”, “a second”, “the first”, or “the second”, may be used merely for the purpose of distinguishing a component from the other components, but do not limit the corresponding components in other aspect (e.g., the importance or the order). It is to be understood that if an element (e.g., a first element) is referred to, with or without the term “operatively” or “communicatively”, as “coupled with,” “coupled to,” “connected with,” or “connected to” another element (e.g., a second element), it means that the element may be coupled with the other element directly (e.g., wiredly), wirelessly, or via a third element.

The term “module” used in the disclosure may include a unit implemented in hardware, software, or firmware and may be interchangeably used with the terms “logic”, “logical block”, “part” and “circuit”. The “module” may be a minimum unit of an integrated part or may be a part thereof. The “module” may be a minimum unit for performing one or more functions or a part thereof. The “module” may include an application-specific integrated circuit (ASIC).

Various embodiments of the disclosure may be implemented by software (e.g., the program140) including an instruction stored in a machine-readable storage medium (e.g., an internal memory136or an external memory138) readable by a machine (e.g., the electronic device101). For example, the processor (e.g., the processor120) of a machine (e.g., the electronic device101) may call the instruction from the machine-readable storage medium and execute the instructions thus called. This means that the machine may perform at least one function based on the called at least one instruction. The one or more instructions may include a code generated by a compiler or executable by an interpreter. The machine-readable storage medium may be provided in the form of non-transitory storage medium. Here, the term “non-transitory”, as used herein, means that the storage medium is tangible, but does not include a signal (e.g., an electromagnetic wave). The term “non-transitory” does not differentiate a case where the data is permanently stored in the storage medium from a case where the data is temporally stored in the storage medium.

The method according to various embodiments disclosed in the disclosure may be provided as a part of a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)) or may be directly distributed (e.g., download or upload) online through an application store (e.g., a Play Store™) or between two user devices (e.g., the smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or generated in a machine-readable storage medium such as a memory of a manufacturer's server, an application store's server, or a relay server.

Each component (e.g., the module or the program) of the above-described components may include one or plural entities. According to various embodiments, at least one or more components of the above components or operations may be omitted, or one or more components or operations may be added. Additionally or alternatively, some components (e.g., the module or the program) may be integrated in one component. In this case, the integrated component may perform the same or similar functions performed by each corresponding components prior to the integration. According to various embodiments, operations performed by a module, a programming, or other components may be executed sequentially, in parallel, repeatedly, or in a heuristic method, or at least some operations may be executed in different sequences, omitted, or other operations may be added.

According to embodiments disclosed in this specification, it is possible to implement a high-resolution image by applying AF information differently and the output of an image sensor depending on camera operation settings and external environmental conditions.

Besides, a variety of effects directly or indirectly understood through the specification may be provided.