Patent Description:
A camera may include a multifocal lens that is able to adjust a distance between lenses included in a body tube and a unifocal lens that is not able to adjust the distance between the lenses included in the body tube. A portable electronic device adopts a camera (hereinafter, referred to as a unifocal camera) including the unifocal lens due to a space constraint.

The portable electronic device may include a single unifocal camera and use a digital zoom function that virtually generates some of image pixels to provide a zoom effect. Alternatively, the portable electronic device may include a plurality of unifocal cameras having different focal distances, and switch a camera providing an image from one unifocal camera to another unifocal camera to provide a zoom effect similar to an optical zoom function.

<CIT> relates to an apparatus and method of generating a zoomed image. <CIT> relates to an apparatus including a pixel array including a plurality of individual pixels grouped into pixel kernels having two or more individual pixels, wherein each pixel kernel includes a floating diffusion electrically coupled to all individual pixels in the kernel.

When an electronic device provides a digital zoom function using a single unifocal camera, an image quality of the electronic device may be lowered after adjustment of zoom setting. In addition, when the electronic device provides a zoom function using a plurality of unifocal cameras, not only a cost of the electronic device and complexity may be increased, but also only a zoom function for a center region of an image sensor may be provided.

Various embodiments disclosed in the disclosure provide a camera module having a multi-cell structure and a portable communication device including the camera module that may lower image resolution degradation when adjusting the zoom setting.

Accordingly, an aspect of the disclosure is to provide a portable communication device according to claim <NUM>.

According to embodiments disclosed in the disclosure, the resolution degradation of the image data due to the zoom setting may be lowered. In addition, various effects that are directly or indirectly grasped through the disclosure may be provided.

In the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

<FIG> illustrates a portable communication device capable of adjusting a zoom magnification according to an embodiment.

Referring to <FIG>, in operation <NUM>, in a preview mode, a portable communication device <NUM> may output, on a display <NUM>, a preview image generated based on data obtained in an entire region of an image sensor included in a camera module <NUM>. The camera module <NUM> may be exposed through a rear surface of the portable communication device <NUM>, for example. For example, the portable communication device <NUM> may obtain data from the image sensor, bin at least a portion of the obtained data to generate first raw image data, and generate first image based on the first raw image data and output the generated first image on the display <NUM>.

When an input associated with adjustment of zoom setting is received in operation <NUM>, the portable communication device <NUM> may output, on the display <NUM>, a preview image generated based on data obtained in the portion of the image sensor in operation <NUM>. The input associated with the adjustment of the zoom setting may include, for example, a pinch-out input in which a space between two fingers, which touched the display <NUM> (e.g., touch screen display), is widened. For example, the portable communication device <NUM> may obtain the data from the portion of the image sensor, re-mosaic at least a portion of the obtained data to generate second raw image data, and generate second image based on the second raw image data and output the generated second image on the display <NUM>. Additionally, or alternatively, the portable communication device <NUM> may store the generated image. For example, the portable communication device <NUM> may store the generated image in response to an input associated with image capturing.

<FIG> illustrates a block diagram of a portable communication device according to an embodiment.

Referring to <FIG>, according to one embodiment, the portable communication device <NUM> may include the camera module <NUM>, the display <NUM>, and a processor <NUM>. In one embodiment, the portable communication device <NUM> may omit some components or further include additional components. For example, the portable communication device <NUM> may further include a memory <NUM>. In one embodiment, some of components of the portable communication device <NUM> may be combined with each other as a single entity, but the single entity may perform the same functions of the corresponding components before the combination.

According to an embodiment, the camera module <NUM> includes an image sensor <NUM> including a plurality of image pixels and a control circuit <NUM> for controlling the image sensor <NUM>. The image sensor <NUM> includes the plurality of image pixels formed in a multi-cell structure in which data for the specified number of image pixels corresponding to the same channel may be binned. For example, the plurality of image pixels are included in a single channel on the NXN pixels basis. To this end, a plurality of ((MXM)/(NXN)) color filters are arranged on the plurality of image pixels, and one color filter is disposed on the NXN pixels. A case in which the NXN pixels are included in the single channel will be described as an example. The NXN pixels corresponding to the same channel may share an output node (e.g., a floating diffusion area). In the disclosure, a case in which the plurality of image pixels is included in the single channel on the 2X2 pixels basis will be described as an example. However, the disclosure is not limited thereto. For example, the plurality of image pixels may be included in the single channel on a matrix form of the 3X3, 4X4, or greater basis. The control circuit <NUM> may use the image sensor <NUM> to obtain image data from at least some of the plurality of image pixels, and process (e.g., bin, re-mosaic) the obtained image data to generate raw image data. For example, the control circuit <NUM> may adjust a timing of obtaining the image data from the plurality of image pixels to bin or not bin the image data.

According to one embodiment, the display <NUM> may display various contents (e.g., a text, an image, a video, an icon, and/or a symbol, or the like), for example. The display <NUM> may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, or an electronic paper display. The display <NUM> may be, for example, a touch screen display capable of detecting a user's touch.

The processor <NUM> may execute operations or data processing associated with control and/or communication of at least one of other components of the portable communication device <NUM> using instructions stored in the memory <NUM>. The processor <NUM> may include at least one of a graphic processing unit (GPU), an application processor, or an image processor.

According to one embodiment, when an input associated with activation of an image capturing function is received, the processor <NUM> may activate the image capturing function of the camera module <NUM>. The input associated with the activation of the image capturing function may be, for example, a user's input of selecting (e.g., touching) an icon associated with the activation of the image capturing function. For example, the processor <NUM> may transmit a request associated with the activation of the image capturing function to the control circuit <NUM> such that driving power is supplied to the camera module <NUM> and the control circuit <NUM> activates the image capturing function of the image sensor <NUM>.

According to one embodiment, when the image capturing function is activated, the processor <NUM> may use the image sensor <NUM> to obtain the first raw image data having a specified channel pattern, which is generated by binning the image data obtained from the first set number (or the specified number) of image pixels corresponding to the same channel with respect to the plurality of image pixels. The first set number may be the total number of the image pixels included in the single channel. The specified channel pattern may be a bayer channel pattern in which R image data and G image data alternately appear on odd row lines, and the G image data and B image data alternately appear on even row lines. For example, the processor <NUM> may use the control circuit <NUM> to control the image sensor <NUM> such that image data for the first set number of the image pixels corresponding to the same channel among the plurality of image pixels are binned and then read-out.

According to one embodiment, the processor <NUM> may display the first image generated based on the first raw image data on the display <NUM>. For example, the processor <NUM> color-interpolates the first raw image data such that each pixel of the first raw image data includes all of R information, G information, and B information, and then converts the color-interpolated first raw image data into a specified format (e.g., a YUV format) to generate the first image data. The processor <NUM> may generate the first image based on the generated first image data and display the first image on the display <NUM>.

According to one embodiment, the processor <NUM> may receive the input associated with the adjustment of the zoom setting while the image capturing function is activated. The input associated with the adjustment of the zoom setting is an input of requesting the adjustment of the zoom magnification, which may include a zoom in and a zoom out. For example, the input associated with the adjustment of the zoom setting may include a first input of widening the space between the two fingers, which touched the touch screen display <NUM>. As another example, the input associated with the adjustment of the zoom setting may include a second input of selecting a first menu of setting a zoom region or a second menu of setting a zoom magnification. In the disclosure, an example in which the input associated with the adjustment of the zoom setting is the first input will be described as an example.

According to one embodiment, when the input associated with the adjustment of the zoom setting is received, the processor <NUM> may determine the zoom magnification based on the received input. For example, the processor <NUM> may determine the zoom magnification based on a change in the space between the two fingers touched the touch screen display <NUM>. In this regard, when the space between the two touched fingers is widened, the processor <NUM> may increase the zoom magnification based on a degree of widening. Further, when the space between the two touched fingers is reduced, the processor <NUM> may decrease the zoom magnification based on a degree of reduction.

According to one embodiment, when the input associated with the adjustment of the zoom setting is received, the processor <NUM> may determine image pixels to be used for obtaining a zoomed image (some of the image pixels) among the plurality of image pixels based on the received input. For example, position information of the some of the image pixels mapped on a selected position (e.g., row information and column information of the some of the image pixels) may be stored in the memory <NUM>. Further, as the processor <NUM> identifies the position information of the some of the image pixels corresponding to the selected position (e.g., a center position between the two fingers) from the memory <NUM>, the processor <NUM> may determine the image pixels to be used for obtaining the zoomed image. The some of the image pixels mapped on the selected position may be different for each zoom magnification. For example, as the zoom magnification is higher, the total number of some of the image pixels may be smaller.

According to one embodiment, when it is identified that the zoom setting (or the zoom magnification) is within a specified first magnification range, the processor <NUM> may use the image sensor <NUM> to obtain the first raw image data generated by binning the image data obtained from the first set number of pixels corresponding to the same channel among the plurality of image pixels, and display the first image generated using the obtained first raw image data on the display <NUM>. The first magnification range may be about <NUM> time, for example.

According to one embodiment, when it is identified that the zoom setting is within a specified second magnification range, the processor <NUM> may use the image sensor <NUM> to obtain the second raw image data generated by re-mosaicing the image data obtained from the some of the plurality of image pixels corresponding to a portion of the image sensor <NUM>. The second magnification range may be, for example, N times corresponding to the number of horizontal image pixels or vertical image pixels included in the single channel. For example, when it is identified that the zoom magnification is within the second magnification range based on the input associated with the adjustment of the zoom setting, the processor <NUM> may identify the position information of the some of the image pixels based on the zoom magnification, which is mapped on the selected position based on the input. The processor <NUM> may use the image sensor <NUM> and the control circuit <NUM> to obtain the image data for the some of the image pixels, and use the control circuit <NUM> to re-mosaic the image data obtained from the some of the image pixels, thereby generating and obtaining the second raw image data having a specified channel pattern. The second raw image data may be image data having the same resolution as the first raw image data.

According to one embodiment, the processor <NUM> may obtain third raw image data instead of the second raw image data when the zoom setting is within the second magnification range, based on at least one of ambient illuminance or a signal to noise ratio (SNR) of the image. For example, when it is identified that the zoom setting is within the second magnification range, the processor <NUM> may identify the ambient illuminance using an illuminance sensor <NUM> and determine whether the ambient illuminance is within a specified illuminance range. The specified illuminance range may be an illuminance range of, for example, equal to or above a reference illuminance (e.g., about <NUM> lux) of determining whether the illuminance is low. When the ambient illuminance is within the specified illuminance range, the processor <NUM> may generate the above-described second raw image data. Further, when the ambient illuminance is not within the specified illuminance range, the processor <NUM> may use the image sensor <NUM> to obtain the third raw image data generated by at least binning image data obtained from the specified number of image pixels corresponding to the same channel among the some of the image pixels corresponding to the portion of the image sensor <NUM>. As another example, when it is identified that the zoom setting is within the second magnification range, the processor <NUM> may identify a signal to noise ratio of the first raw image data and determine whether the signal to noise ratio is within a specified ratio range. The specified ratio range may be determined experimentally as a criterion for determining a deterioration of an image quality of the first raw image data. When the signal to noise ratio is within the specified ratio range, the processor <NUM> may use the image sensor <NUM> to obtain the above-described second raw image data. In various embodiments, the processor <NUM> may determine whether to obtain the second raw image data even after, based on the signal to noise ratio of the obtained second raw image data. When the signal to noise ratio is not within the specified ratio range, the processor <NUM> may use the image sensor <NUM> to obtain the third raw image data generated by binning the data obtained from the specified number of image pixels corresponding to the same channel among the some of the plurality of image pixels corresponding to the portion of the image sensor <NUM>. As another example, when it is identified that the zoom setting is within the second magnification range, the processor <NUM> may determine whether the ambient illuminance is within the specified illuminance range and whether the signal to noise ratio of the first image is within the specified ratio range. When the ambient illuminance is within the specified illuminance range and the signal to noise ratio of the first image is within the specified ratio range, the processor <NUM> may use the image sensor <NUM> to obtain the above-described second raw image data. When the ambient illuminance is not within the specified illuminance range, or when the signal to noise ratio of the first image is not within the specified ratio range, the processor <NUM> may use the image sensor <NUM> to obtain the above-described third raw image data. The processor <NUM> may display the second image generated based on the second raw image data or the third image generated based on the third raw image data on the display <NUM>. According to the embodiment described above, as the processor <NUM> displays a third image (a preview image) using the third raw image data generated by binning the image data for the some of the image pixels instead of the second raw image, the processor <NUM> may display a preview image having a higher image quality based on a surrounding situation or an image quality.

According to one embodiment, the processor <NUM> may store the preview image (one of the first to third images) in the memory <NUM>. In addition, the processor <NUM> may store the image generated in response to the input associated with the image capturing in the memory <NUM>.

According to various embodiments, when the ambient illuminance is not within the specified illuminance range or the signal to noise ratio of the image is not within the specified ratio range, the processor <NUM> may output a screen indicating that the adjustment of the zoom magnification is not available in response to the input associated with the adjustment of the zoom setting and not adjust the zoom magnification, or provide a digital zoom based on the first raw image data or the second raw image data.

According to various embodiments, based on the zoom setting being within a specified third magnification range, the processor <NUM> may use the image sensor <NUM> to obtain fourth raw image data in which image data obtained from pixels of the second set number, which is different from the first set number, corresponding to the same channel among the plurality of image pixels is at least binned. The second set number may be the number of some of the image pixels included in each channel. The third magnification range may include a magnification of greater than about <NUM> time and less than N times. For example, when the plurality of image pixels is included in a single channel on the 3X3 pixels basis, the second magnification range may include magnification of about <NUM> times, and the third magnification range may include magnification of about <NUM> times. Additionally or alternatively, the processor <NUM> may use the image sensor <NUM> to obtain the fourth raw image data, which is generated by binning the image data obtained from the second set number of pixels corresponding to the same channel with respect to some of the remaining image pixels among the plurality of image pixels and further re-mosaicing the binned image data. The processor <NUM> may display fourth image generated using the obtained fourth raw image data on the display <NUM>.

According to the above embodiment, in the portable communication device <NUM>, a resolution of the raw image data after the adjustment of the zoom setting (e.g., the second raw image data) may not be lowered compared to that of the raw image data before the adjustment of the zoom setting (e.g., the first raw image data) through the binning or the re-mosaicing of the image data for the plurality of image pixels of the image sensor <NUM>.

According to one embodiment, a portable communication device (e.g., the portable communication device <NUM> of <FIG>) may include a camera module (the camera module <NUM> of <FIG>) including an image sensor (e.g., the image sensor <NUM> of <FIG>) including a plurality of image pixels and a control circuit (e.g., the control circuit <NUM> of <FIG>) for controlling the image sensor; a display (e.g., the display <NUM> of <FIG>); and a processor (e.g., the processor <NUM> of <FIG>). The processor may activate an image capturing function of the camera module, when the image capturing function is activated, use the image sensor to obtain first raw image data having a specified channel pattern, which is generated by binning image data obtained from the specified number of image pixels corresponding to the same channel with respect to the plurality of image pixels, display a first image generated based on the first raw image data on the display, receive an input associated with adjustment of zoom setting while the image capturing function is activated, use the image sensor to obtain second raw image data having the specified channel pattern, which is generated by re-mosaicing image data obtained from some of the plurality of image pixels corresponding to a portion of the image sensor, based at least on the input, and display a second image generated based on the second raw image data on the display.

The processor may identify a position selected as a zoom region and zoom magnification based at least on the input associated with the adjustment of the zoom setting, and the some of the image pixels may be image pixels corresponding to the identified zoom magnification, mapped on the selected position.

The second raw image data may be raw image data generated to have a resolution corresponding to a resolution of the first raw image data.

The processor may identify an ambient illuminance and obtain the second raw image data using the image sensor when the ambient illuminance is within a specified illuminance range.

When the ambient illuminance is not within the specified illuminance range, the processor may use the image sensor to obtain third raw image data generated by at least binning the image data obtained from the specified number of image pixels corresponding to the same channel among the some of the plurality of image pixels corresponding to the portion of the image sensor, and display third image generated based on the third raw image data on the display.

The processor may identify a signal to noise ratio of the first image and, when the signal to noise ratio of the first image is within a specified ratio range, obtain the second raw image data using the image sensor.

When the signal to noise ratio is not within the specified ratio range, the processor may use the image sensor to obtain the third raw image data generated by binning the data obtained from the specified number of image pixels corresponding to the same channel among the some of the plurality of image pixels corresponding to the portion of the image sensor, and display the third image generated based on the third raw image data on the display.

According to one embodiment, a portable communication device (e.g., the portable communication device <NUM> of <FIG>) may include a camera module (the camera module <NUM> of <FIG>) including an image sensor (e.g., the image sensor <NUM> of <FIG>) including a plurality of image pixels and a control circuit (e.g., the control circuit <NUM> of <FIG>) for controlling the image sensor; a display (e.g., the display <NUM> of <FIG>); and a processor (e.g., the processor <NUM> of <FIG>). The processor may receive an input associated with adjustment of zoom setting associated with capturing, use the image sensor to obtain first raw image data generated by binning image data obtained from the first set number of pixels corresponding to the same channel with respect to the plurality of image pixels based on the zoom setting being within a specified first magnification range, display a first image generated using the obtained first raw image data on the display, use the image sensor to obtain second raw image data generated by re-mosaicing image data obtained from some of the plurality of image pixels corresponding to a portion of the image sensor based on the zoom setting being within a specified second magnification range, and display a second image generated using the obtained second raw image data on the display.

Based on the zoom setting being within a specified third magnification range, the processor may use the image sensor to obtain third raw image data in which image data obtained from pixels of the second set number, which is different from the first set number, corresponding to the same channel is at least binned with respect to the plurality of image pixels, and display a third image generated using the obtained third raw image data on the display.

The processor may use the image sensor to obtain the third raw image data, which is generated by binning the image data obtained from the second set number of pixels corresponding to the same channel with respect to the plurality of image pixels and then further re-mosaicing the binned image data.

Based on the zoom setting being within the specified third magnification range, the processor may use the image sensor to obtain the third raw image data in which the image data obtained from the second set number of pixels corresponding to the same channel among some of the remaining image pixels corresponding to a remaining portion of the image sensor among the plurality of image pixels is at least binned.

The processor may identify information associated with a position and a magnification of a zoom region based on the input associated with the adjustment of the zoom setting, and the portion of the image sensor may be a portion corresponding to the position and the magnification of the zoom region.

The processor may identify ambient illuminance, when the ambient illuminance is within a specified illuminance range, use the image sensor to obtain the second raw image data having the specified channel pattern, which is generated by re-mosaicing the image data obtained from the some of the plurality of image pixels corresponding to the portion of the image sensor, when the ambient illuminance is not within the specified illuminance range, use the image sensor to obtain the third raw image data generated by binning the image data obtained from the specified number of image pixels corresponding to the same channel among the some of the plurality of image pixels corresponding to the portion of the image sensor, and display the third image generated based on the third raw image data on the display.

The processor may identify a signal to noise ratio of the first image, when the signal to noise ratio is within a specified ratio range, use the image sensor to obtain the second raw image data having the specified channel pattern, which is generated by re-mosaicing the image data obtained from the some of the plurality of image pixels corresponding to the portion of the image sensor, when the signal to noise ratio is not within the specified ratio range, use the image sensor to obtain the third raw image data generated by binning the data obtained from the specified number of image pixels corresponding to the same channel among some of the plurality of image pixels corresponding to the portion of the image sensor, and display the third image generated based on the third raw image data on the display. <FIG> illustrates a configuration diagram of a camera module according to an embodiment.

Referring to <FIG>, the camera module <NUM> may include the image sensor <NUM> and the control circuit <NUM>. In one embodiment, the camera module <NUM> may omit some components or further include additional components. In one embodiment, some of components of the camera module <NUM> may be combined with each other as a single entity, but the single entity may perform the same functions of the corresponding components before the combination.

According to one embodiment, the image sensor <NUM> may include a plurality of image pixels 113a and a read-out circuit 113b. A charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor may be included. A plurality of image pixels arranged in an MXM matrix is formed in a single channel on the NXN pixels basis. To this end, a plurality ((MXM)/(NXN)) of color filters are arranged on the plurality of image pixels, and a single color filter is disposed on the NXN pixels. For example, <NUM> (=<NUM>/<NUM>) color filters (R color filter, G color filter, or B color filter) are arranged on a plurality of image pixels arranged in an 8X8 matrix. Each of the <NUM> (=<NUM>/<NUM>) color filters may be disposed to have a specified channel pattern on 2X2 pixels. The NXN pixels included in the same channel may be formed in a multi-cell structure capable of binning image data for at least some pixels. For example, the NXN pixels corresponding to the same channel may share an output node (e.g., a floating diffusion area).

According to one embodiment, the control circuit <NUM> includes a timing controller 115a, a row selector 115b, a column selector 115c, an analog digital convertor (ADC) 115d, an RGB converter 115e, and an output buffer 115f. The timing controller 115a may generate a control signal for controlling an operation of at least one of the row selector 115b, the column selector 115c, the ADC 115d, the RGB converter 115e, and the output buffer 115f. The row selector 115b may selectively activate one of row lines of the image sensor <NUM> based on the control signal of the timing controller 115a. The column selector 115c may selectively activate one of column lines of the image sensor <NUM> based on the control signal of the timing controller 115a. The ADC 115d may convert analog image data output from the image sensor <NUM> into digital image data. The RGB converter 115e may re-mosaic digital image data to correspond to a specified channel pattern, thereby generating the raw image data having the specified channel pattern. The specified channel pattern may be the bayer channel pattern in which the R image data and the G image data alternately appear on the odd row lines, and the G image data and the B image data alternately appear on the even row lines. The output buffer 115f may buffer the raw image data, for example, on a frame basis in the re-mosaic process. The control circuit <NUM> to be described in a following document represents the timing controller 115a and each component controlled by the timing controller 115a.

According to one embodiment, when receiving a request associated with the activation of the image capturing function from an external processor (e.g., the processor <NUM> of <FIG>), the control circuit <NUM> may activate the image capturing function of the image sensor <NUM>. For example, the timing controller 115a may supply the driving power to the image sensor <NUM> to activate the image capturing function of the image sensor <NUM>.

According to one embodiment, when the image capturing function of the image sensor <NUM> is activated, the control circuit <NUM> may use the image sensor <NUM> to generate first raw image data having a specified channel pattern by binning image data obtained from the first set number of image pixels corresponding to the same channel with respect to the plurality of image pixels. The first set number may be, for example, the total number of image pixels included in the same channel. For example, the timing controller 115a may control the row selector 115b and the column selector 115c to transmit the image data for the first set number of image pixels individually (or simultaneously) to a floating diffusion area, and use the read-out circuit 113b to read-out the floating diffusion area after all of the image data (charges) for the first set number of image pixels are transmitted to the floating diffusion area, thereby binning (e.g., summing up) the image data for the first set number of image pixels. The timing controller 115a may digitally convert the binned analog image data using the ADC 115d and buffer the digital data on a frame basis using the output buffer 115f to generate the first raw image data.

According to one embodiment, the control circuit <NUM> may receive a request associated with zoom setting from the processor <NUM> while the image capturing function is activated. When the request associated with the zoom setting is received, the control circuit <NUM> may identify some of the plurality of image pixels to read-out the image data based on the request. The control circuit <NUM> may obtain image data for the identified some of the image pixels, and re-mosaic the obtained image data to generate second raw image data having a specified channel pattern. The second raw image data may be image data having the same resolution as the first raw image data or image data having a resolution higher than that of the first raw image data. For example, the timing controller 115a may use the row selector 115b and the column selector 115c to read-out each of the analog image data for the some of the image pixels of the image sensor <NUM>, and use the ADC 115d to digitally convert each of the read-out analog image data. The timing controller 115a may use the RGB converter 115e to re-mosaic the (digital) image data for the some of the image pixels based on the specified channel pattern, and buffer the re-mosaiced image data on a frame basis using the output buffer 115f, thereby generating the second raw image data having the specified channel pattern.

According to one embodiment, the control circuit <NUM> may transmit the first raw image data or the second raw image data obtained using the image sensor <NUM> to the processor <NUM>. For example, the timing controller 115a may transmit the first raw image data or the second raw image data buffered on a frame basis by the output buffer 115f to the processor <NUM>.

According to various embodiments, the control circuit <NUM> may generate third raw image data by binning an image obtained from the second set number of image pixels corresponding to the same channel among the some of the image pixels based on zoom magnification and then re-mosaicing the binned image. For example, the control circuit <NUM> may generate the third raw image data having the specified channel pattern by binning the image data obtained from the second set number of image pixels corresponding to the same channel with respect to some of the remaining image pixels among the plurality of image pixels based on zoom magnification being within a third magnification range, and re-mosaic the binned image data. The second set number may be smaller than the total number of image pixels included in the same channel. As another example, the timing controller 115a may use the image sensor <NUM> to individually (or simultaneously) transmit, to a floating diffusion area, the image data for the second set number of image pixels included in the same channel among the some of the remaining image pixels, and use the read-out circuit 113b to read-out the floating diffusion area after the image data (charges) for the second set number of image pixels are transmitted to the floating diffusion area, thereby binning (e.g., summing up) the image data for the second set number of image pixels. The timing controller 115a may use the ADC 115d to digitally convert each of the read-out analog image data. The timing controller 115a may use the RGB converter 115e to re-mosaic the (digital) image data for the some of the remaining image pixels based on a specified channel pattern, and use the output buffer 115f to buffer the re-mosaiced image data on a frame basis, thereby generating the third raw image data having the specified channel pattern.

According to various embodiments, each channel may include the NXK (K is a natural number different from N) number of image pixels.

<FIG> illustrates image pixels included in a single channel according to an embodiment.

Referring to <FIG>, each of image pixels PD1, PD2, PD3, and PD4 113a may be a photodiode capable of accumulating light received after being reflected on an external object. A first end of each of the image pixels PD1, PD2, PD3, and PD4 may be connected to the ground. Further, second ends of the image pixels PD1, PD2, PD3, and PD4 may be electrically connected to transfer transistors TR1, TR2, TR3, and TR4, respectively.

First ends of the transfer transistors (e.g., TR1) may be connected to the image pixels (e.g., PD1), respectively. Further, second ends of the transfer transistors (e.g., TR1) may be connected to a common node (output node) CN. The image pixels PD1, PD2, PD3, and PD4 included in the same channel may pass through the common node CN and share a floating diffusion area FD. The floating diffusion area FD may be electrically connected to the common node CN to accumulate electric charges output to the common node CN therein. The transfer transistors TR1, TR2, TR3, and TR4 may sequentially and respectively output electric charges stored in the image pixels to the common node CN.

For example, when the zoom magnification is within the first magnification range, the read-out circuit (e.g., the read-out circuit 113b of <FIG>) may transfer the electric charges (binned image data) accumulated in the floating diffusion area FD to the control circuit <NUM> after all electric charges accumulated in all image pixels (of the first set number) included in the same channel are transferred to the common node CN. Alternatively, for example, when the zoom magnification is within the second magnification range, the read-out circuit 113b may transfer the electric charges accumulated in the floating diffusion area FD to the control circuit <NUM> after an electric charge accumulated in one image pixel is transferred to the common node CN and before an electric charge accumulated in another image pixel is transferred to the common node CN. Further, for example, when the zoom magnification is within the third magnification range, the read-out circuit 113b may transfer the electric charges accumulated in the floating diffusion area FD to the control circuit <NUM> when electric charges accumulated in the second set number of image pixels among the image pixels included in each channel are transferred to the common node CN.

According to one embodiment, a camera module (the camera module <NUM> of <FIG>) includes an image sensor (e.g., the image sensor <NUM> of <FIG>) including a plurality of image pixels (e.g., the plurality of image pixels 113a of <FIG>) and a control circuit (e.g., the control circuit <NUM> of <FIG>) for controlling the image sensor. The control circuit may activate an image capturing function of the image sensor, when the image capturing function is activated, use the image sensor to generate first raw image data having a specified channel pattern, which is generated by binning image data obtained from the specified number of image pixels corresponding to the same channel with respect to the plurality of image pixels, transmit the generated first raw image data to an external processor (e.g., the processor <NUM> of <FIG>) electrically connected to the control circuit, receive a request associated with zoom setting while the image capturing function is activated, use the image sensor to generate second raw image data having the specified channel pattern by re-mosaicing image data for some of the plurality of image pixels corresponding to a portion of the image sensor based at least on the request, and transmit the generated second raw image data to the external processor.

The control circuit may generate the first raw image data or the second raw image data based on a specified resolution.

The control circuit may identify position information of the some of the image pixels based on the request associated with the zoom setting and obtain image data from the some of the image pixels corresponding to the identified position information.

The control circuit may identify information associated with zoom magnification based on the request associated with the zoom setting, generate second raw image data having the specified channel pattern by re-mosaicing image data for some of the plurality of image pixels corresponding to a portion of the image sensor based on the zoom magnification being within a first magnification range, transmit the second raw image data to the external processor, generate third raw image data having the specified channel pattern by binning image data obtained from image pixels of the different specified number corresponding to the same channel with respect to some of the remaining image pixels based on the zoom magnification being within a second magnification range, and then re-mosaicing the image data, and transmit the third raw image data to the external processor.

The plurality of image pixels includes NXN image pixels included in each channel. N is a natural number equal to or greater than <NUM>. Further, the specified number may be the total number of image pixels included in each channel, and the different specified number may be the number of some of the total image pixels included in each channel.

<FIG> illustrates a process of generating first raw image data according to an embodiment.

Referring to <FIG>, an image sensor (e.g., the image sensor <NUM> of <FIG>) may include a plurality of image pixels (e.g., 113a of <FIG>) arranged in an 8X8 matrix. The plurality of image pixels 113a may be included in a single channel (e.g., <NUM>) on the 2X2 pixels basis. For example, one color filter (R color filter, G color filter, or B color filter) may be disposed on the 2X2 pixels.

According to one embodiment, the control circuit <NUM> generates first raw image data <NUM> having a specified channel pattern by binning image pixels included in the same channel with respect to the plurality of image pixels based on a command (e.g., a request associated with activation of the image sensor) of the processor <NUM>. For example, the control circuit <NUM> may bin image data obtained from four image pixels corresponding to a R channel <NUM> to generate one R image data <NUM>, and bin image data obtained from <NUM> image pixels corresponding to a G channel <NUM> to generate one G image data <NUM>. The control circuit <NUM> may bin image data obtained from four image pixels corresponding to a R channel <NUM> to generate one R image data <NUM>, and bin image data obtained from <NUM> image pixels corresponding to a G channel <NUM> to generate one G image data <NUM>. The control circuit <NUM> may bin image data obtained from four image pixels corresponding to a G channel <NUM> to generate one G image data <NUM>, and bin image data obtained from <NUM> image pixels corresponding to a B channel <NUM> to generate one B image data <NUM>. The control circuit <NUM> may bin image data obtained from four image pixels corresponding to a G channel <NUM> to generate one G image data <NUM>, and bin image data obtained from <NUM> image pixels corresponding to a B channel <NUM> to generate one B image data <NUM>. Similarly, the control circuit <NUM> may bin image data for four image pixels corresponding to the same channel for the remaining channels, and as a result, the image sensor <NUM> may generate the first raw image data <NUM> having the specified channel pattern. The specified channel pattern may be a bayer channel pattern in which R image data and the G image data alternately appear on odd row lines, and the G image data and the B image data alternately appear on even row lines.

<FIG> illustrates a process of generating second raw image data according to an embodiment.

Referring to <FIG>, the control circuit <NUM> generates second raw image data <NUM> having the specified channel pattern by re-mosaicing image data <NUM> for some of the plurality of image pixels corresponding to a portion of the image sensor <NUM> based on a command (e.g., a request associated with zoom setting) of the processor <NUM> while the image capturing function is activated. For example, the control circuit <NUM> may identify position information of the some of the image pixels based on the request associated with the zoom setting, and obtain the image data <NUM> from the some of the image pixels based on the identified position information. As the control circuit <NUM> performs re-mosaic of exchanging image data that do not correspond to the specified channel pattern among the image data <NUM> obtained from the image sensor <NUM> with each other, the control circuit <NUM> generates the second raw image data <NUM> having the specified channel pattern.

Referring to <FIG> and <FIG>, according to the above-described embodiment, the first raw image data <NUM> and the second raw image data <NUM> may have the same resolution.

<FIG> illustrates an example of a method for controlling an image sensor by a processor (e.g., the processor <NUM> of <FIG>) according to an embodiment.

Referring to <FIG>, in operation <NUM>, the processor <NUM> may activate the image capturing function of the camera module <NUM>. For example, when the input associated with the activation of the image capturing function is received, the processor <NUM> may activate the image capturing function of the camera module <NUM>. The input associated with the activation of the image capturing function may be, for example, the user's input of selecting (e.g., touching) the icon associated with the activation of the image capturing function.

In operation <NUM>, when the image capturing function is activated, the processor <NUM> may use the image sensor <NUM> to obtain the first raw image data having the specified channel pattern, which is generated by binning the image data obtained from the specified number (the first set number) of image pixels corresponding to the same channel with respect to the plurality of image pixels. The specified number may be, for example, the total number of image pixels included in each channel. The specified channel pattern may be the bayer channel pattern.

In operation <NUM>, the processor <NUM> may display the first image generated based on the first raw image data on the display <NUM>. For example, the processor <NUM> may color-interpolate the first raw image data such that each pixel of the first raw image data includes the R information, the G information, and the B information, and generate the
first image by converting the color-interpolated first raw image data into the specified format (e.g., YUV format).

In operation <NUM>, the processor <NUM> may receive the input associated with the adjustment of the zoom setting while the image capturing function is activated. For example, the input associated with the adjustment of the zoom setting may include the first input of widening the space between the two fingers, which touched the touch screen display <NUM>.

In operation <NUM>, the processor <NUM> may use the image sensor <NUM> to obtain the second raw image data having the specified channel pattern, which is generated by re-mosaicing the image data obtained from the some of the plurality of image pixels corresponding to the portion of the image sensor <NUM> based at least on the input. For example, the processor <NUM> may identify the selected position and the zoom magnification based on the input associated with the adjustment of the zoom setting, and identify the position information of the some of the image pixels corresponding to the zoom magnification mapped on the selected position. The processor <NUM> may use the image sensor <NUM> to obtain the image data of the some of the image pixels of the identified position information, and use the control circuit <NUM> to re-mosaic the obtained image data to correspond to the specified channel pattern, thereby obtaining the second raw image data.

In operation <NUM>, the processor <NUM> may display the second image generated based on the second raw image data on the display <NUM>. For example, the processor <NUM> may color-interpolate the second raw image data such that each pixel of the second raw image data includes the R information, the G information, and the B information, and generate the second image by converting the color-interpolated second raw image data into the specified format (e.g., YUV format).

<FIG> illustrates another example of a method for controlling an image sensor by the processor <NUM> according to an embodiment.

In operation <NUM>, the processor <NUM> may receive the input associated with the adjustment of the zoom setting associating with the capturing. For example, the processor <NUM> may receive the input associated with the adjustment of the zoom setting while the image capturing function of the camera module <NUM> is activated. The input associated with the adjustment of the zoom setting may be, for example, the input of requesting the adjustment of the zoom magnification (e.g., zoom in or zoom out).

In operation <NUM>, the processor <NUM> may use the image sensor <NUM> to obtain the first raw image data generated by binning the image data obtained from the first set number of pixels corresponding to the same channel with respect to the plurality of image pixels based on the zoom setting being within the specified first magnification range. The processor <NUM> may display the first image generated using the obtained first raw image data on the display <NUM>. The first set number may be the total number of image pixels corresponding to the same channel. The first magnification range may be about <NUM> time, for example.

In operation <NUM>, the processor <NUM> may use the image sensor <NUM> to obtain the second raw image data generated by re-mosaicing the image data obtained from the some of the plurality of image pixels corresponding to the portion of the image sensor <NUM> based on the zoom setting being within the specified second magnification range. The processor <NUM> may display the second image generated using the obtained second raw image data on the display <NUM>. For example, the second magnification range may be, for example, a multiple corresponding to the number of horizontal image pixels N or the number of vertical image pixels N included in each channel. For example, the processor <NUM> may identify the selected position and the zoom magnification based on the input associated with the adjustment of the zoom setting, and identify the position information of the some of the image pixels corresponding to the zoom magnification mapped on the selected position. The processor <NUM> may use the image sensor <NUM> to obtain the image data of the some of the image pixels of the identified position information, and use the control circuit <NUM> to re-mosic the obtained image data to correspond to the specified channel pattern, thereby obtaining the second raw image data. The second raw image data may be image data having the same resolution as the first raw image data.

According to various embodiments, in operation <NUM>, when the zoom setting is within the specified second magnification range, the processor <NUM> may use the image sensor <NUM> to re-mosaic the image data obtained from the plurality of image pixels to obtain the raw image data, select the raw image data corresponding to the portion of the image sensor <NUM> from the obtained raw image data, and use the selected image data to generate the second image.

<FIG> illustrates a method for controlling an image sensor by the control circuit <NUM> according to an embodiment.

In operation <NUM>, the control circuit (e.g., the control circuit <NUM> of <FIG>) may activate the image capturing function of the image sensor <NUM>. When the request associated with the activation of the image capturing function is received from the processor <NUM>, the control circuit <NUM> may activate the image capturing function of the image sensor <NUM>.

In operation <NUM>, when the image capturing function is activated, the control circuit <NUM> may use the image sensor <NUM> to generate the first raw image data having the specified channel pattern, which is generated by binning the image data obtained from the specified number of image pixels corresponding to the same channel with respect to the plurality of image pixels. The specified number may be, for example, the total number of image pixels corresponding to the same channel. The specified channel pattern may be the bayer channel pattern.

In operation <NUM>, the control circuit <NUM> may transmit the first raw image data to the external processor (e.g., the processor <NUM> of <FIG>) electrically connected to the control circuit.

In operation <NUM>, the control circuit <NUM> may receive the request associated with the zoom setting while the image capturing function is activated.

In operation <NUM>, the control circuit <NUM> may use the image sensor <NUM> to re-mosaic the image data for the some of the plurality of image pixels corresponding to the portion of the image sensor based at least on the request, thereby generating the second raw image data having the specified channel pattern. The second raw image data may have the same resolution as the first raw image data.

In operation <NUM>, the control circuit <NUM> may transmit the generated second raw image data to the external processor <NUM>.

<FIG> illustrates a block diagram of an electronic device <NUM> in a network environment <NUM> according to various embodiments. According to an embodiment, the electronic device <NUM> (e.g., the communication device <NUM> of <FIG>) may include a processor <NUM>, memory <NUM> (e.g., the memory <NUM> of <FIG>), an input device <NUM>, a sound output device <NUM>, a display device <NUM> (e.g., the display <NUM> of <FIG>), an audio module <NUM>, a sensor module <NUM> (e.g., the illuminance sensor <NUM> of <FIG>), an interface <NUM>, a haptic module <NUM>, a camera module <NUM> (e.g., the camera module <NUM> of <FIG>), a power management module <NUM>, a battery <NUM>, a communication module <NUM>, a subscriber identification module(SIM) <NUM>, or an antenna module <NUM>.

The program 940may be stored in the memory <NUM> as software, and may include, for example, an operating system (OS) <NUM>, middleware <NUM>, or an application <NUM>.

A corresponding one of these communication modules may communicate with the external electronic device via the first network <NUM> (e.g., a short-range communication network, such as Bluetooth<IMG>, wireless-fidelity (Wi-Fi) direct, or infrared data association (IrDA)) or the second network <NUM> (e.g., a long-range communication network, such as a cellular network, the Internet, or a computer network (e.g., LAN or wide area network (WAN)).

<FIG> illustrates a block diagram <NUM> of the camera module <NUM> according to various embodiments. Referring to <FIG>, the camera module <NUM> may include a lens assembly <NUM>, a flash <NUM>, an image sensor <NUM> (e.g., the control circuit <NUM> and the image sensor <NUM> of <FIG>), an image stabilizer <NUM>, memory <NUM> (e.g., buffer memory), or an image signal processor <NUM> (e.g., the processor <NUM> of <FIG>). The lens assembly <NUM> may collect light emitted or reflected from an object whose image is to be taken. The lens assembly <NUM> may include one or more lenses. According to an embodiment, the camera module <NUM> may include a plurality of lens assemblies <NUM>. In such a case, the camera module <NUM> may form, for example, a dual camera, a <NUM>-degree camera, or a spherical camera. Some of the plurality of lens assemblies <NUM> may have the same lens attribute (e.g., view angle, focal length, auto-focusing, f number, or optical zoom), or at least one lens assembly may have one or more lens attributes different from those of another lens assembly. The lens assembly <NUM> may include, for example, a wide-angle lens or a telephoto lens.

The flash <NUM> may emit light that is used to reinforce light reflected from an object. According to an embodiment, the flash <NUM> may include one or more light emitting diodes (LEDs) (e.g., a red-green-blue (RGB) LED, a white LED, an infrared (IR) LED, or an ultraviolet (UV) LED) or a xenon lamp. The image sensor <NUM> may obtain an image corresponding to an object by converting light emitted or reflected from the object and transmitted via the lens assembly <NUM> into an electrical signal. According to an embodiment, the image sensor <NUM> may include one selected from image sensors having different attributes, such as a RGB sensor, a black-and-white (BW) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same attribute, or a plurality of image sensors having different attributes. Each image sensor included in the image sensor <NUM> may be implemented using, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

The image stabilizer <NUM> may move the image sensor <NUM> or at least one lens included in the lens assembly <NUM> in a particular direction, or control an operational attribute (e.g., adjust the read-out timing) of the image sensor <NUM> in response to the movement of the camera module <NUM> or the electronic device <NUM> including the camera module <NUM>. This allows compensating for at least part of a negative effect (e.g., image blurring) by the movement on an image being captured. According to an embodiment, the image stabilizer <NUM> may sense such a movement by the camera module <NUM> or the electronic device <NUM> using a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module <NUM>. According to an embodiment, the image stabilizer <NUM> may be implemented, for example, as an optical image stabilizer.

The memory <NUM> may store, at least temporarily, at least part of an image obtained via the image sensor <NUM> for a subsequent image processing task. For example, if image capturing is delayed due to shutter lag or multiple images are quickly captured, a raw image obtained (e.g., a Bayer-patterned image, a high-resolution image) may be stored in the memory <NUM>, and its corresponding copy image (e.g., a low-resolution image) may be previewed via the display device <NUM>. Thereafter, if a specified condition is met (e.g., by a user's input or system command), at least part of the raw image stored in the memory <NUM> may be obtained and processed, for example, by the image signal processor <NUM>. According to an embodiment, the memory <NUM> may be configured as at least part of the memory <NUM> or as a separate memory that is operated independently from the memory <NUM>.

The image signal processor <NUM> may perform one or more image processing with respect to an image obtained via the image sensor <NUM> or an image stored in the memory <NUM>. The one or more image processing may include, for example, depth map generation, three-dimensional (3D) modeling, panorama generation, feature point extraction, image synthesizing, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor <NUM> may perform control (e.g., exposure time control or read-out timing control) with respect to at least one (e.g., the image sensor <NUM>) of the components included in the camera module <NUM>. An image processed by the image signal processor <NUM> may be stored back in the memory <NUM> for further processing, or may be provided to an external component (e.g., the memory <NUM>, the display device <NUM>, the electronic device <NUM>, the electronic device <NUM>, or the server <NUM>) outside the camera module <NUM>. According to an embodiment, the image signal processor <NUM> may be configured as at least part of the processor <NUM>, or as a separate processor that is operated independently from the processor <NUM>. If the image signal processor <NUM> is configured as a separate processor from the processor <NUM>, at least one image processed by the image signal processor <NUM> may be displayed, by the processor <NUM>, via the display device <NUM> as it is or after being further processed.

According to an embodiment, the electronic device <NUM> may include a plurality of camera modules <NUM> having different attributes or functions. In such a case, at least one of the plurality of camera modules <NUM> may form, for example, a wide-angle camera and at least another of the plurality of camera modules980 may form a telephoto camera. Similarly, at least one of the plurality of camera modules <NUM> may form, for example, a front camera and at least another of the plurality of camera modules980 may form a rear camera.

It should be appreciated that various embodiments of the present disclosure and the terms used therein are not intended to limit the technological features set forth herein to particular embodiments and include various changes as long as they are within the scope of the invention as defined by the appended claims.

The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or be distributed (e.g., downloaded or uploaded) online via an application store (e.g., PlayStore<IMG>), or between two user devices (e.g., smart phones) directly.

Claim 1:
A portable communication device (<NUM>, <NUM>) comprising:
a camera module (<NUM>, <NUM>) including an image sensor (<NUM>, <NUM>) including a plurality of image pixels (113a) and a control circuit (<NUM>, <NUM>) configured to control the image sensor (<NUM>, <NUM>), wherein the plurality of image pixels are arranged in an MXM matrix and a plurality of colour filters are arranged on the plurality of image pixels such that a single colour filter is arranged on NXN pixels which are included in a same colour channel, N being a natural number equal to or greater than <NUM> and smaller than M and a number of colour filters being equal to (MXM)/(NXN);
a display (<NUM>, <NUM>); and
a processor (<NUM>, <NUM>) configured to:
activate an image capturing function of the camera module (<NUM>, <NUM>);
based on the image capturing function being activated, obtain first raw image data comprising a specified color channel pattern generated by binning image data obtained from a specified number of image pixels among the NXN pixels corresponding to a same color channel, by using the image sensor (<NUM>, <NUM>), wherein the specified colour channel pattern corresponds to a specific arrangement of colour filters repeating on the image sensor including Red, Green, Blue filters;
display a first image generated based on the first raw image data on the display (<NUM>, <NUM>);
while the image capturing function is activated, receive an input associated with an adjustment of a zoom setting;
obtain second raw image data comprising the specified color channel pattern, generated by re-mosaicing image data obtained from some of the plurality of image pixels corresponding to a portion of the image sensor selected based on the input, by using the image sensor, wherein the re-mosaicing comprises exchanging with each other image data among the obtained image data corresponding to said portion that do not correspond to the specified color channel pattern to generate the specified colour channel pattern; and
display a second image generated based on the second raw image data on the display (<NUM>, <NUM>).