Video signal processing apparatus, video signal processing method, and imaging device

Disclosed is a video signal processing apparatus applicable to various LCD panels. This video signal processing apparatus includes a format converting unit which converts first-format image data into second-format image data, a point sequential converting unit which converts the second-format image data into point sequenced data, a resizing unit which resizes the point sequenced data, a line memory which stores the resized point sequenced data, and an output control unit which controls to continuously read out the point sequenced data from the line memory.

FIELD OF THE INVENTION

This invention relates to a video signal processing technique of displaying image data and the like.

BACKGROUND OF THE INVENTION

An imaging device such as a digital camera generally has an LCD panel and can display a photographed image on this LCD panel. Since the size of a photographed image and the number of pixels of the LCD panel are not equal in most cases, a resizing process is necessary.

A device which outputs display data and a pulse skipped clock to an LCD controller is described in Japanese Patent Laid-Open No. 2002-305752. Note that the display data is intermittently formed from resized image data which is formed by resizing image data obtained from an imaging element in accordance with the display size of a TV monitor.

Unfortunately, some general LCD controllers cannot accept the pulse skipped clock and intermittent display data as described above. Accordingly, there are needs for a video signal processing apparatus usable by such LCD controllers.

Therefore, the feature of the present invention is to solve this problem and at least one of several other problems. Note that the other problems will be understood through the whole of this specification.

SUMMARY OF THE INVENTION

A video signal processing apparatus according to the present invention comprises, e.g., a format converting unit, point sequential converting unit, resizing unit, line memory, and output control unit. The format converting unit converts first-format image data into second-format image data. The point sequential converting unit converts the second-format image data into point sequenced data. The resizing unit resizes the point sequenced data. The line memory stores the resized point sequenced data. The output control unit controls to continuously read out the point sequenced data from the line memory.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

First Embodiment

FIG. 1is a block diagram of an imaging device including a video signal processing apparatus according to the first embodiment. This imaging device is, e.g., a digital camera or digital video camera which incorporates a display device and can be connected to an external TV monitor.

InFIG. 1, reference numeral100denotes an imaging element which converts an optical image into an electrical signal;101, an A/D converter which converts the analog image signal from the imaging element100into a digital signal;102, an imaging signal processor which performs, e.g., gamma processing, interpolation, and matrix transformation on the output data from the A/D converter101, thereby forming YUV-format image (video) data;103, a resizing circuit which resizes the image data from the imaging signal processor102into the display size of a TV monitor; and104, a memory I/F which writes and reads out image data and various control data in and from a memory105.

Reference numeral106denotes a buffering FIFO memory which converts the image data read out from the memory I/F104into a different data rate;107, an NTSC/PAL encoder which converts the YUV-format image data read out from the FIFO memory into a composite video signal;108, a D/A converter; and109, a TV monitor. Note that as indicated by the dotted-line boundary, the TV monitor109need not be mounted on the video signal processing apparatus or imaging device. In this case, the TV monitor109is connected to the video signal processing apparatus or imaging device via a video signal terminal or the like.

Reference numeral110denotes a YUV/RGB converter which converts the YUV-format image data into an RGB format;111, a point sequencer which point sequences the output RGB signal from the YUV/RGB converter110into RGB. Point sequencing is a concept opposite to plane sequencing. Point sequencing gives importance to pixels forming an image, and sequentially records them from the upper left corner to the lower right corner of the image. For example, a signal is recorded like RGBRGBRGB . . . . Note that plane sequencing records a signal like RRRRRR . . . , GGGGGG . . . , and BBBBB . . . . Reference numeral112denotes a linear interpolation resizing circuit which enlarges/reduces the RGB point sequenced signal.

Reference numeral113denotes a line memory which stores the resized RGB point sequenced signal;114, an LCD controller which processes the RGB point sequenced signal continuously output from the line memory113; and115, an LCD panel which displays images and the like under the control of the LCD controller114. Note that it is also possible to use, instead of the LCD panel115, another type of display device such as a self-emitting display device which can be mounted on the imaging device.

This embodiment will be explained by assuming that the display size of the TV monitor109is 720 horizontal pixels×480 vertical lines of the NTSC system, and that the LCD panel115has 480 horizontal pixels×240 lines or 960 horizontal pixels×240 lines. In the following explanation, image data stored in the memory105is displayed on the TV monitor109and LCD panel115at the same time.

The output analog image signal from the imaging element100is converted into a digital signal (image data) by the A/D converter101. This image data is input to the imaging signal processor102. The imaging signal processor102performs, e.g., gamma processing, interpolation, and matrix transformation on the input image data, thereby generating YUV-format image data. The resizing circuit103enlarges or reduces (i.e., resizes) the formed image data in at least one of the horizontal direction and vertical direction. This resizing forms display image data corresponding to 720 horizontal pixels×480 vertical lines as the display size of the TV monitor109. After that, the display image data is stored in the memory105via the memory I/F104.

Generally, an NTSC TV monitor performs interlaced display. In the first field, therefore, only 240 odd-numbered lines (a first-field image) of the 480 lines forming the display image are readout from the memory105and displayed on the TV monitor109. In the second field, only 240 even-numbered lines (a second-field image) are read out from the memory105and displayed on the TV monitor109.

On the other hand, the LCD panel115performs progressive display. In this embodiment, the number of vertical lines is 240, i.e., half that of the TV monitor109. Also, in this embodiment, first-field images and second-field images are alternately displayed in synchronism with the TV monitor109. In this manner, the display image data read out from the memory105is displayed on the TV monitor109and LCD panel115at the same time.

A display timing controller120reads out the display image data stored in the memory105via the memory I/F104, and sequentially stores the readout data in the FIFO memory106. In synchronism with the TV display timing, the display timing controller120reads out the image data from the FIFO memory106, and supplies the readout data to the NTSC/PAL encoder107and YUV/RGB converter110.

The NTSC/PAL encoder107generates a composite video signal from the YUV-format image data read out from the FIFO memory106. The D/A converter108converts the composite video signal into an analog video signal. The TV monitor109displays an image on the basis of the analog video signal.

On the other hand, the YUV/RGB converter110converts the YUV signal into an RGB signal in accordance with the following equations. That is, the equations of conversion from an RGB signal to a YUV signal are
Y=0.299R+0.587G+0.114B
U=0.493(B−Y)
V=0.877(R−Y)
Therefore, the inverse conversion is
R=Y+(1/0.877)V=Y+(292/256)V
B=Y+(1/0.493)U=Y+(519/256)U
G=Y−(0.299/0.587)(292/256)V−(0.114/0.587)(519/256)U
=Y−(149V+101U)/256

When a write enable signal (WR_ENABLE) to the line memory113is High, a write controller130of the line memory113writes a calculation result Z of the linear interpolation resizing circuit112into the line memory113. A read controller140continuously reads out data from the line memory113, and outputs the readout data to the LCD controller114. That is, the read controller140is an example of an output control unit for point sequenced data.

FIG. 2is a circuit diagram showing details of the RGB point sequencer and linear interpolation resizing circuit according to the embodiment. The operations of these circuits will be explained below with reference toFIG. 2. InFIG. 2, reference numerals201,202, and203denote flip-flops for respectively delaying R, G, and B input data by one cycle;204, a selector which selects one of the RGB input data; and205, a selector which selects one of the RGB input data delayed by one cycle. By inputting a select signal to the selectors204and205, input data of two consecutive points having the same color can be output to the linear interpolation resizing circuit112on the subsequent stage.

The linear interpolation resizing circuit112includes an interpolation coefficient calculator207, linear interpolation calculator208, and flip-flops209and210. When the number of horizontal pixels of the LCD panel115is smaller than 720 as the number of horizontal pixels of the TV monitor, the linear interpolation resizing circuit12performs reduction processing. When the number of horizontal pixels of the LCD panel115is larger than 720 as the number of horizontal pixels of the TV monitor, the linear interpolation resizing circuit12performs enlargement processing.

(1) Reduction Processing

FIG. 3is a view showing timings when the reduction ratio is ⅔. As an example, a case in which the number of horizontal pixels of the LCD panel115is 720×⅔=480 will be explained below. It is readily understood fromFIG. 3that two output data (OUT_DATA) are generated from three input data (IN_DATA) by an interpolation coefficient k. Also, the resized image data (OUT_DATA) are continuously output to the line memory113in synchronism with a VALID signal. In this way, reduction to ⅔ is achieved.

More specifically, the interpolation coefficient calculator207sequentially outputs the interpolation coefficient k to the linear interpolation calculator208. The interpolation coefficient calculator207also sequentially outputs the VALID signal indicating the flag of valid output data to the flip-flop210and the like. When the resizing rate is ⅔, for example, the VALID signal changes to High twice in three cycles. The interpolation coefficient k is, e.g., a signal which repeats 0, ½, 0, ½, . . . .

For input data of two consecutive points (the nth data is X(n) and the (n+1)th data is X(n+1) where n is a natural number) input from the RGB point sequencer111, the linear interpolation calculator208calculates a linear interpolation value Z by using the interpolation coefficient k and the following equation.
Z=X(n)×(1−k)+X(n+1)×k

FIG. 4is a view showing the positional relationship between the linear interpolation value Z output from the linear interpolation calculator according to the embodiment, and X(n) and X(n+1) input from the RGB point sequencer. As shown inFIG. 4, the position at which the interval between X(n) and X(n+1) is divided into k:1−k corresponds to Z.

When VALID signal=High, the linear interpolation value Z is input to the flip-flop209. The linear interpolation value Z is output as RGB point sequenced data. Note that a signal obtained by delaying the VALID signal by one cycle in the flip-flop210is the write enable signal (WR_ENABLE) to the line memory (to be described later).

(2) Enlargement Processing

The linear interpolation resizing circuit112executes the enlargement processing at a frequency twice the operating frequency range of the reduction processing. This means that the input data undergoes twofold oversampling (enlargement). The linear interpolation resizing circuit112executes the reduction processing by using this sampling data. If the desired enlargement ratio is P (1<P<2), the reduction ratio of the interpolation coefficient calculator207need only be set at P/2. As a consequence, it is possible to attain an enlargement ratio of 2×P/2=P as a total.

For example, when data is input at a data rate of 13.5 MHz from the RGB point sequencer111, the linear interpolation resizing circuit112operates at 27 MHz as a double frequency. Accordingly, the RGB point sequenced data is reduced after oversampling is performed at a double frequency (i.e., 27 MHz) of the data rate. In this case, an interpolation coefficient K of the linear interpolation calculator208is calculated as follows from the interpolation coefficient k output from the interpolation coefficient calculator207and the VALID signal indicating the flag of valid output data.

First, assuming that the nth data is X(n) and the (n+1)th data is X(n+1), data represented by
Y={X(n)+X(n+1)}/2
is virtually inserted in the center between the nth and (n+1)th data by twofold oversampling.

The interpolation coefficient calculator207operates at the double frequency. Therefore, the timing at which the VALID signal changes to High when the interpolation calculation is performed between X(n) and Y (in the case of phase 1) differs from that when the interpolation calculation is performed between Y and X(n+1) (in the case of phase 2).

When the interpolation calculation is performed between X(n) and Y (in the case of phase 1), the output data Z is

When the interpolation calculation is performed between Y and X(n+1) (in the case of phase 2), the output data Z is

Accordingly, the interpolation coefficient K of the linear interpolation calculator208need only be set to
K=k/2
or
K=(1+k)/2
in accordance with the phase in which the VALID signal changes to High.

FIGS. 5A and 5Bare views showing the positional relationships between X(n), X(n+1), and Z in the different phases according to the embodiment.FIG. 5Ashows the positional relationship between X(n), X(n+1), and Z in phase 1.FIG. 5Bshows the positional relationship between X(n), X(n+1), and Z in phase 2.

FIG. 6is a view showing timings when the enlargement ratio is 4/3. In this example, it is assumed that the number of horizontal pixels of the LCD panel115is 720× 4/3=960. The reduction ratio of the interpolation coefficient calculator207is set at ⅔. Also, the interpolation coefficient k repeats 0, ½, 0, ½, . . . , in the same manner as shown inFIG. 3. However, the interpolation coefficient K repeats 0, ¼, ½, and ¾ in accordance with the phase of the VALID signal (FIG. 6).

The linear interpolation calculator208calculates the linear interpolation value Z by using the data (X(n) and X(n+1)) of two consecutive points input from the RGB point sequencer111and the interpolation coefficient K.
Z=X(n)×(1−K)+X(n+1)×K

When VALID signal=High, the linear interpolation value Z is input to the flip-flop209, and output as RGB point sequenced data to the line memory113. A signal obtained by delaying the VALID signal by one cycle in the flip-flop210is the write enable signal (WR_ENABLE) to the line memory113(to be described later).

When the write enable signal (WR_ENABLE) to the line memory113is High, the write controller130of the line memory113writes the linear interpolation value Z of the linear interpolation resizing circuit112into the line memory113. The read controller140continuously reads out data from the line memory113, and outputs the readout data to the LCD controller114. Finally, the image is displayed on the LCD panel115.

In the video signal processing apparatus according to this embodiment, the line memory113for storing point sequenced data is placed after the RGB point sequencer111, so point sequenced data can be continuously output. Accordingly, it is possible to provide a video signal processing apparatus suitably usable in an LCD controller which cannot accept any pulse skipped clock or intermittent display data.

Note that it is of course also possible to provide a video signal processing method which converts first-format image data into second-format image data, converts the second-format image data into point sequenced data, resizes the point sequenced data, stores the resized point sequenced data in the line memory113, and performs output control by continuously reading out the point sequenced data from the line memory113.

Also, the capacity of the line memory113for storing the point sequenced data can be reduced by placing the line memory113after the RGB point sequencer111. That is, when each of Y, U, V, R, G, and B is, e.g., 8-bit data, a memory capacity of 16 bits per pixel is necessary if the data is stored by the YUV422 format. However, since the RGB point sequenced data is stored, the necessary memory capacity is 8 bits per pixel, i.e., the memory capacity can be reduced to half that when data is stored by the YUV422 format.

In addition, the video signal processing apparatus according to this embodiment includes the NTSC/PAL encoder107which generates a composite video signal from YUV-format image data. Therefore, images can be displayed not only on the LCD panel115but also on the external TV monitor109. For example, the LCD panel115of an imaging device such as a digital camera is effective to monitor images during photographing, but inconvenient for a large number of people to appreciate images because the display size is small. Accordingly, images of an imaging device are desirably displayed on the TV monitor109via a video output terminal or the like.

Furthermore, in this embodiment, the operating frequency range of the linear interpolation resizing circuit112is suitably changed in accordance with whether the reduction processing or enlargement processing is performed, and the interpolation coefficient used in the linear interpolation calculator208is also suitably determined. This avoids a large increase in circuit scale of the linear interpolation resizing circuit112.

Second Embodiment

In this embodiment, a technique which reduces the necessary storage capacity of a line memory113by delaying the read start timing of the line memory113from its write start timing by a time corresponding to the enlargement ratio/reduction ratio will be explained. Note that the explanation will be simplified by denoting the already explained portions by the same reference numerals as above.

(1) Reduction Processing

FIG. 7is a graph showing examples of the timings of a write pointer and read pointer for the line memory. Referring toFIG. 7, the reduction ratio is ⅔. For example, the number of horizontal pixels of original image data is 720, and the number of horizontal pixels of an LCD panel115is 480. The processing shown inFIG. 7is particularly set such that the read pointer (READ POINTER) points the first data after the write pointer (WRITE POINTER) points the 160th data.

In this embodiment, for example, a read controller140starts reading out data from the line memory113after the (720−480=240)th data is input to a linear interpolation resizing circuit112, and the (240×⅔=160)th data is written in the line memory113. Accordingly, data read can be executed without outpacing the written data. In this case, the line memory113need only have a storage capacity capable of storing data of at least 160 pixels.

Generally, the necessary storage capacity of the line memory113can be calculated as a function of the number x (x<720) of horizontal pixels of the LCD panel115.

Since the reduction ratio is x/720, (720−x) data are input to the linear interpolation resizing circuit112.

If the read controller140starts data read after
(720−x)×(x/720)
data are written in the line memory113, data read can be performed without outpacing the written data.

Accordingly, the minimum necessary storage capacity of the line memory113is
(720−x)×(x/720)

FIG. 8is a graph showing the relationship between the number of horizontal pixels of the LCD panel and the necessary capacity of the line memory in the reduction processing according to the embodiment. Referring toFIG. 8, the x-axis represents the number of horizontal pixels of the LCD panel115, and the y-axis represents the necessary capacity of the line memory113.FIG. 8shows that if the line memory113has a storage capacity of at least 180 data, the video signal processing apparatus according to this embodiment can be applied to an LCD panel having 720 horizontal pixels or less.

(2) Enlargement Processing

FIG. 9is a view showing examples of the timings of the write pointer and read pointer for the line memory when the enlargement ratio is 4/3 (when the number of horizontal pixels of the LCD panel is 960).

As is apparent fromFIG. 9, the read controller140preferably starts reading out data after 720×2−960=480 data are input to the linear interpolation resizing circuit112, and 480×⅔=320 data are written in the line memory113. In this manner, data can be read out without outpacing the written data. The line memory need only have a capacity of at least 320 data.

The necessary storage capacity of the line memory113can be calculated as a function of the number x (720<x<1,440) of horizontal pixels of the LCD panel115. The enlargement ratio can be represented by (x/720). Accordingly, data read need only be started after (720×2−x) data are input to the linear interpolation resizing circuit112, and (1,440−x)×(x/1,440) data are written in the line memory113. That is, data can be read out without outpacing the written data.

As described above, the necessary storage capacity of the line memory113is
(1,440−x)×(x/1,440)

FIG. 10is a graph showing the relationship between the number of horizontal pixels of the LCD panel and the necessary capacity of the line memory in the enlargement processing according to the embodiment. The x-axis represents the number of horizontal pixels of the LCD panel. The y-axis represents the necessary capacity of the line memory. It will be understood fromFIG. 10that if the line memory113has a storage capacity of at least 360 data, the video signal processing apparatus according to this embodiment can be applied to an LCD panel having 720 to 1,440 horizontal pixels.

In this embodiment as explained above, the necessary storage capacity of the line memory113can be reduced by delaying the read start timing of the line memory113from its write start timing by a time corresponding to the enlargement ratio/reduction ratio. It is also possible to obtain high display quality with little deterioration in the enlargement processing as well.

Note that in the explanation of each of the above embodiments, the display size of the TV monitor109is 720 horizontal pixels×480 vertical lines. Note also that in the above explanation, the number of pixels of the LCD panel115is 480 horizontal pixels×240 lines or 960 horizontal pixels×240 lines. However, these practical numerical values are of course mere examples.

For example, the expressions indicated by
(720−x)×(x/720)
(1,440−x)×(x/1,440)
which represent the storage capacity of the line memory113can be generalized into
(y−x)(x/y)
where y indicates the number of horizontal pixels of an original image (the number of horizontal pixels of the TV monitor109) in the reduction processing, and the number of horizontal pixels of a maximally enlarged image in the enlargement processing. As is apparent from this expression, the storage capacity of the line memory113can be made smaller than the half of the data amount of one line of YUV-format data.

Note that in the above embodiments, YUV-format image (video) data and RGB-format image data are explained. However, the present invention is of course also applicable to image data having another format.

The present invention can be applied to a system constituted by a plurality of devices, or to an apparatus comprising a single device. Furthermore, it goes without saying that the invention is applicable also to a case where the object of the invention is attained by supplying a program to a system or apparatus.

This application claims the benefit of Japanese Application No. 2005-202101, filed Jul. 11, 2005, which is hereby incorporated by reference herein in its entirety.