Patent Description:
Recently, much research has been conducted on three-dimensional (3D) image-related techniques. Also, devices for implementing high-quality holograms in real time by using a complex spatial modulator (SLM) capable of controlling the amplitude and phase of light at the same time have been studied actively.

Computer-generated holograms (CGH) is used to reproduce holographic moving images, and an image processing apparatus calculates hologram values corresponding to positions on a hologram plane, which requires a huge computational amount. That is, the image processing apparatus needs to perform a Fourier transform once to express a point in a space, and has to perform a Fourier transform as many times as the number of pixels to express an image in the space.

An image processing apparatus such as a television (TV), a mobile device, etc., may process image data to reproduce holographic images. In this case, the image processing apparatus performs a Fourier transform with respect to image data and reproduces an image based on the transformed data.

When the image processing apparatus performs the Fourier transform, the computational amount is quite large and much time is required. In particular, as a portable device such as a mobile device has a limited size and power, there is a need for methods for reducing the computational amount and time for performing a Fourier transform.

United States Patent Number <CIT> presents a two-dimensional (2D) fast Fourier transform (FFT) converter which utilises parallel FFT processors and a frame serial processing path to convert 2D images of large pixel count and high dynamic range at a rate equal to or exceeding real-time processing rate of <NUM> frames per second.

United States Patent Application Publication Number <CIT> relates to a concept for carrying out a FFT on a block of sample points by executing a 1D FFT on all vertical lines of sample point and then executing a 1D FFT on each resulting horizontal line of transformed data.

Provided is a method and apparatus for performing Fourier transform with respect to image data.

According to an aspect of the invention, there is provided an image processing apparatus for performing two-dimensional (2D) fast Fourier transform (FFT) with respect to image data according to claim <NUM>.

According to another aspect of the invention, there is provided an image processing method of performing 2D FFT with respect to image data according to claim <NUM>.

In this regard, the present example embodiments may have different forms and should not be construed as being limited to the descriptions set forth herein.

Hereinafter, example embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<FIG> is a flowchart illustrating a process of processing image data according to an example embodiment. Referring to <FIG>, an image processing apparatus may receive image data and outputs an image.

In operation <NUM>, the image processing apparatus may receive image data. In an example ebmodiment, when a layer-based algorithm is applied to image data in computer-generated holography (CGH) computation, the image data may include color data (or color images), depth data (or depth images), and the like. The color data may indicate a plurality of colors for each plane. In an example embodiment, the color data may include a red image, a blue image, and a green image. A layer-based algorithm is used to divide a reproduction region of a hologram according to a depth and process data of each plane obtained by the division. The image processing apparatus may perform a Fourier transform or an inverse Fourier transform with respect to data of each plane obtained by the division and may generate a holographic image.

In operation <NUM>, the image processing apparatus may perform image-quality correction and field calculation. The image processing apparatus may correct the image data to improve the quality of the image data.

In operation <NUM>, the image processing apparatus may perform a Fourier transform or a fast Fourier transform (FFT). In an example embodiment, the image processing apparatus may perform a Fourier transform with respect to image data in a two-dimensional (2D) matrix form. The image processing apparatus may perform one-dimensional (1D) Fourier transform twice for 2D Fourier transform. The image processing apparatus may perform a 1D Fourier transform with respect to image data in a row direction and may perform a 1D Fourier transform with respect to the transformed image data in a column direction. The image processing apparatus may generate a holographic image through a Fourier transform.

The image processing apparatus may include a plurality of cores which perform a Fourier transform with respect to the image data in parallel. For example, the image processing apparatus may allocate image data of each plane to a plurality of cores which may perform a Fourier transform with respect to the allocated image data.

The process of performing a Fourier transform with respect to image data by the image processing apparatus will be described in detail with reference to <FIG> and <FIG> according to an example embodiment.

In operation <NUM>, the image processing apparatus may perform pixel encoding. The image processing apparatus may generate data to be input to a screen through pixel encoding.

In operation <NUM>, the image processing apparatus may output an image to an image display apparatus.

The image processing apparatus may perform a Fourier transform with respect to a part of data in operation <NUM>, which requires many calculations, thereby reducing the camputional amount. The image processing apparatus may store a part of data in operation <NUM>, thus reducing the amount of data stored in a memory.

The image processing apparatus may generate one new frame by combining a color frame with a depth frame. The image processing apparatus may alternately dispose a line of the color frame and a line of the depth frame.

The image processing apparatus may process image data and depth data line-by-line to reduce the number of frames stored in an external memory.

<FIG> is a flowchart illustrating a process of transforming data according to an example embodiment. Referring to <FIG>, the image processing apparatus or a Fourier transform apparatus may perform a 1D FFT with respect to image data <NUM> twice and may generate final data <NUM> (primary 2D FFT). In an example embodiment, the image processing apparatus may perform a 1D FFT with respect to the image data <NUM> once in a column direction to generate intermediate data <NUM>, and may perform a 1D FFT with respect to the intermediate data <NUM> in a row direction to generate the final data <NUM>. Secondary 2D FFT may also be performed by performing the 1D FFT twice. The primary 2D FFT may be a FFT from a pupil to a retina, and the secondary 2D FFT may be a FFT from a panel to the pupil.

The primary 2D FFT and the secondary 2D FFT may have orders of performing 1D FFT that are reverse with respect to each other. For example, if the 1D FFT is performed twice in the column direction and then in the row direction when the primary 2D FFT is performed, the 1D FFT may be performed twice in the row direction and then in the column direction when the secondary 2D FFT is performed.

In <FIG>, an example has been described in which the image processing apparatus performs the 1D FFT first in the column direction. On the other hand, in <FIG>, an example will be described in which the image processing apparatus performs the 1D FFT first in the row direction.

<FIG> and <FIG> illustrate only a case the primary 2D FFT is performed, and the secondary 2D FFT may also be performed identically to the primary 2D FFT or in orders of the row direction and the column direction, which are reverse to the primary 2D FFT.

The image processing apparatus performs the 1D FFT with respect to the image data <NUM> in the column direction. The intermediate data <NUM> indicates a result of performing the 1D FFT with respect to the image data <NUM> in the column direction. An arrow expressed in the image data <NUM> indicates a direction in which the image processing apparatus performs the 1D FFT. A straight line expressed in the intermediate data <NUM> indicates a direction in which the image data <NUM> is transformed.

The image processing apparatus reads the stored intermediate data <NUM> from the memory and performs the 1D FFT with respect to the intermediate data <NUM> in the row direction. The image processing apparatus may read the intermediate data <NUM> in the row direction from the memory and output the read intermediate data <NUM> to each 1D FFT processor.

The image processing apparatus performs the 1D FFT with respect to the intermediate data <NUM> in the row direction and generates the final data <NUM>. The final data <NUM> is a result of performing the 1D FFT with respect to the image data <NUM> in the column direction and in the row direction.

<FIG> is a flowchart illustrating a process of transforming data according to another embodiment. Referring to <FIG>, the image processing apparatus or the Fourier transform apparatus performs the 1D FFT with respect to the image data <NUM> twice and generates the final data <NUM>. For example, the image processing apparatus may perform the 1D FFT with respect to the image data <NUM> once in the row direction to generate intermediate data <NUM>, and may perform the 1D FFT with respect to the intermediate data <NUM> in the column direction to generate the final data <NUM>. In <FIG> and <FIG>, the orders of the column and the row are reversed, and matters applied in <FIG> may be identically applied to <FIG>.

<FIG> is a block diagram of an image processing apparatus according to an embodiment. Referring to <FIG>, an image processing apparatus <NUM> may include a camera <NUM>, a processor <NUM>, and a memory <NUM>. The image processing apparatus <NUM> may be an electronic device such as a computer, a mobile device, a display device, a wearable device, a digital camera, etc., or a central processing unit (CPU), a graphic processing unit (GPU), or the like.

The camera <NUM> captures an image and obtains a color image and a depth image. The color image and the depth image are obtained in the unit of a frame. The color image may include a red image, a green image, and a blue image. Each of the red image, the green image, and the blue image is one frame. The depth image is obtained for each color. That is, the camera <NUM> obtains a depth image for the red image, a depth image for the green image, and a depth image for the blue image. The depth image for each color is also one frame.

The processor <NUM> generates a frame in which a frame of the color image and a frame of the depth image are combined. The combined frame may be one frame. For example, the processor <NUM> may generate a first frame by combining the red image with the depth image for the red image, a second frame by combining the green image with the depth image for the green image, and a third frame by combining the blue image with the depth image for the blue image.

The memory <NUM> stores the color images and the depth images. The memory <NUM> stores the frames generated by the processor <NUM>.

<FIG> is a block diagram of an image processing apparatus according to another embodiment. An image processing apparatus <NUM> may be a mobile device, a display device, a wearable device, a CPU, a GPU, or the like.

The image processing apparatus <NUM> may include a controller <NUM>, a first core <NUM>, a memory <NUM>, and a second core <NUM>. The memory <NUM> may be a dynamic random access memory (DRAM) or a synchronous random access memory (SRAM).

The controller <NUM> controls the first core <NUM>, the memory <NUM>, the second core <NUM>, and so forth. The controller <NUM> determines data input to the first core <NUM> and the second core <NUM>. The controller <NUM> designates an operation to be performed by the first core <NUM> and the second core <NUM>. For example, the controller <NUM> may control the first core <NUM> to perform the 1D FFT in the row direction and the second core <NUM> to perform the 1D FFT in the column direction. The controller <NUM> stores data generated during Fourier transform in the memory <NUM>.

The controller <NUM> controls the first core <NUM> and the second core <NUM> to perform the primary 2D FFT and the secondary 2D FFT with respect to image data. The primary 2D FFT includes performing the 1D FFT twice and the secondary 2D FFT includes performing the 1D FFT twice. The controller <NUM> may control data input to the first core <NUM> and the second core <NUM> to perform the 2D FFT twice (that is, to perform the primary 2D FFT and the secondary 2D FFT). The controller <NUM> controls a data flow such that the first core <NUM> and the second core <NUM> perform the secondary 2D FFT after performing the primary 2D FFT. Thus, the image processing apparatus <NUM> may perform the primary 2D FFT and the secondary 2D FFT (perform the 1D FFT a total of four times) by using the two cores <NUM> and <NUM>. For example, the controller <NUM> may input the primary 2D FFT-transformed data to the first core <NUM> and input data output from the first core <NUM> to the second core <NUM> for the secondary 2D FFT. The controller <NUM> may input the primary 2D FFT-transformed data to the second core <NUM> and input data output from the second core <NUM> to the first core <NUM> for the secondary 2D FFT. Once the primary 2D FFT-transformed data is input to the second core <NUM>, a process of storing and reading the primary 2D FFT-transformed data in and from the memory <NUM> may be omitted.

The first core <NUM> performs Fourier transform with respect to data included in each line of a frame. For example, the first core <NUM> may perform the 1D FFT with respect to a frame in the row direction. One row or one column may be referred to as one line. When the first core <NUM> performs the 1D FFT with respect to the frame in the row direction, it means that the first core <NUM> performs the 1D FFT with respect to pixel values included in each row of the frame.

The first core <NUM> outputs the data to the memory <NUM>. The first core <NUM> outputs a result of performing the 1D FFT to the memory <NUM> each time when the result is generated.

The first core <NUM> may include a plurality of 1D FFT processors. The plurality of 1D FFT processors may perform the 1D FFT with respect to each line of the frame. For example, the number of 1D FFT processors included in the first core <NUM> may be a divisor of the number of total rows of the frame. If the number of rows (lines) of the frame is <NUM>, the first core <NUM> may include <NUM>, <NUM>, <NUM>, or <NUM>1D FFT processors. If the first core <NUM> includes <NUM>1D FFT processors, the first core <NUM> may perform the 1D FFT with respect to <NUM> lines at the same time.

The memory <NUM> may store and output data. The memory <NUM> may be a DRAM or a SRAM.

The second core <NUM> may perform the 1D FFT with respect to data. The second core <NUM> may include a plurality of 1D FFT processors. The plurality of 1D FFT processors may perform the 1D FFT with respect to each column of the frame. For example, the number of 1D FFT processors included in the second core <NUM> may be a divisor of the number of total columns of the frame. If the number of columns of the frame is <NUM>, the second core <NUM> may include <NUM>, <NUM>, <NUM>, or <NUM>1D FFT processors. If the second core <NUM> includes <NUM>1D FFT processors, the second core <NUM> may perform the 1D FFT with respect to <NUM> columns at the same time.

<FIG> is a block diagram of an image processing apparatus according to an embodiment. Referring to <FIG>, an image processing apparatus <NUM> may include a first core <NUM>, a first buffer <NUM>, a memory <NUM>, a demultiplexer <NUM>, a second buffer <NUM>, and a second core <NUM>.

With reference to <FIG>, a case will be described as an example in which a frame size is <NUM> x <NUM>. In other words, when the size of a frame is <NUM> x <NUM>, one line of the frame includes <NUM> pixel values and the frame includes a total of <NUM> rows. That is, the frame includes <NUM> columns and <NUM> rows. Thus, the 1D FFT processors included in the first core <NUM> are <NUM>-point FFT processors, and the 1D FFT processors included in the second core <NUM> are <NUM>-point FFT processors. The <NUM>-point FFT processors perform Fourier transform with respect to <NUM> pixel values, and the <NUM>-point FFT processors perform Fourier transform with respect to <NUM> pixel values. The number of processors included in each of the first core <NUM> and the second core <NUM> may vary with an input frame.

The first core <NUM> may include a plurality of <NUM>-point FFT processors. The plurality of <NUM>-point FFT processors may perform the 1D FFT with respect to each line of the frame. For example, the first core <NUM> may include <NUM><NUM>-point FFT processors. A <NUM>-point FFT processor <NUM> indicates a <NUM>th processor, a <NUM>-point FFT processor <NUM> indicates a <NUM>st processor, and a <NUM>-point FFT processor <NUM> indicates a <NUM>st processor. The <NUM>-point FFT processor <NUM> performs Fourier transform with respect to a first line (or a first row) of the frame, the <NUM>-point FFT processor <NUM> performs Fourier transform with respect to a second line of the frame, and the <NUM>-point FFT processor <NUM> performs Fourier transform with respect to a <NUM>st line of the frame.

The <NUM>-point FFT processors <NUM> through <NUM> may perform Fourier transform with respect to respective lines at the same time and output intermediate values. The intermediate value indicates a pixel value generated by performing Fourier transform with respect to the frame.

The first buffer <NUM> sequentially stores an intermediate value output from the first core <NUM>. The first buffer <NUM> stores intermediate values output from the <NUM>-point FFT processors <NUM> through <NUM>. For example, the first buffer <NUM> may store <NUM> intermediate values sequentially output from the <NUM>-point FFT processors <NUM> through <NUM> and output <NUM> intermediate values to the memory <NUM>. <NUM> intermediate values stored first indicate pixel values of the first column of intermediate data. Next, the first buffer <NUM> may store <NUM> intermediate values sequentially output from the <NUM>-point FFT processors <NUM> through <NUM>, and <NUM> intermediate values stored second indicate pixel values of the second column of the intermediate value.

The memory <NUM> may store the intermediate data. The memory <NUM> sequentially stores an intermediate value output from the first buffer <NUM>. While a description has been made of an example in which the memory <NUM> stores data received from the first buffer <NUM> in <FIG>, the memory <NUM> may store data output from the second core <NUM>.

The demultiplexer <NUM> reads data from the memory <NUM> and outputs the read data to the second buffer <NUM>. The demultiplexer <NUM> reads data in a direction in which the second core <NUM> performs the 1D FFT. For example, when the second core <NUM> performs the 1D FFT in the column direction, the demultiplexer <NUM> may read data stored in the memory <NUM> in the column direction. The demultiplexer <NUM> outputs the read data to the second buffer <NUM>.

The second buffer <NUM> may include a plurality of buffers. Each buffer outputs data to each <NUM>-point FFT processor of the second core <NUM>. The plurality of buffers may store one line. When the second core <NUM> performs the 1D FFT in the column direction, one buffer may store data included in one column. Thus, the second buffer <NUM> may store <NUM> lines.

The second core <NUM> may include a plurality of <NUM>-point FFT processors. The plurality of <NUM>-point FFT processors may perform the 1D FFT with respect to each line of the frame. "<NUM>-point" indicates a processor that performs the 1D FFT with respect to <NUM> pixel values of the frame. For example, the second core <NUM> may include <NUM><NUM>-point FFT processors. A <NUM>-point FFT processor <NUM> indicates a <NUM>th processor, a <NUM>-point FFT processor <NUM> indicates a <NUM>st processor, and a <NUM>-point FFT processor <NUM> indicates a <NUM>st processor. The <NUM>-point FFT processor <NUM> performs Fourier transform with respect to a first line (or a first row) of the frame, the <NUM>-point FFT processor <NUM> performs Fourier transform with respect to a second line of the frame, and the <NUM>-point FFT processor <NUM> performs Fourier transform with respect to a <NUM>st line of the frame. The <NUM>-point FFT processors <NUM> through <NUM> may perform Fourier transform with respect to respective lines at the same time and output intermediate values. The intermediate value indicates a pixel value generated by performing Fourier transform with respect to the frame, and indicates a part of the intermediate data.

Although not shown in <FIG>, the image processing apparatus <NUM> may further include a plurality of operators. The plurality of operators may perform a focus term operation and a depth summation operation. The plurality of operators may be connected with <NUM>-point processors included in the second core <NUM>, like the second buffer <NUM>. One operator sums plural data according to a depth. For example, if a depth level is <NUM>, the operator sums sequentially output <NUM> lines to generate one line, thereby performing depth summation.

<FIG> is a view for describing a data processing sequence according to an embodiment. Referring to <FIG>, the image processing apparatus <NUM> performs the 2D FFT twice by using the two cores <NUM> and <NUM>, and inputs the depth-summed data to the second core <NUM> instead of being input to the memory <NUM> to reduce the number of times data is stored in the memory <NUM>.

With reference to <FIG>, a description will be made of an example in which the first core <NUM> performs the 1D FFT in the row direction and the second core <NUM> performs the 1D FFT in the column direction.

First, data is input to the first core <NUM>. The first core <NUM> performs the 1D FFT with respect to data in the row direction.

Second, the first core <NUM> outputs the 1D FFT-transformed data to the memory <NUM> through the first buffer <NUM>.

Third, the second core <NUM> reads data from the memory <NUM> in the column direction through the demultiplexer <NUM> and the second buffer <NUM>. The second core <NUM> performs the 1D FFT with respect to data. The second core <NUM> performs the 1D FFT in the column direction.

Fourth, the second core <NUM> outputs data to operators <NUM>.

Fifth, the operators <NUM> perform the focus term operation and the depth summation operation.

Sixth, the operators <NUM> output the depth-summed data to the second core <NUM>.

In the embodiment of <FIG>, when the primary 2D FFT is performed, the first core <NUM> performs the 1D FFT prior to the second core <NUM>. When the secondary 2D FFT is performed, the second core <NUM> performs the 1D FFT prior to the first core <NUM>. Data output from the operators <NUM> is data read in the column direction, and thus, when the data is output to the second core <NUM>, the data is not necessarily stored in the memory <NUM> before being output. To output the data output from the operators <NUM> to the first core <NUM>, the data needs to be read in the row direction, such that the data output from the operators <NUM> has to be read in the row direction after being stored in the memory <NUM>. Thus, by inputting the primary FFT-transformed data output from the second core <NUM> to the second core <NUM>, a process of storing the primary FFT-transformed data in the memory <NUM> may be omitted.

Seventh, the second core <NUM> outputs the 1D FFT-transformed data to the memory <NUM> in the column direction. The 1D FFT-transformed data is data generated during the secondary 2D FFT.

Eighth, the first core <NUM> reads the data that is 1D-FFT transformed by the second core <NUM> from the memory <NUM>. The first core <NUM> performs the 1D FFT with respect to the read data. The first core <NUM> performs the 1D FFT in the row direction.

Ninth, the first core <NUM> outputs the 2D FFT-transformed data.

<FIG> is a view for describing a data processing sequence according to another embodiment. Referring to <FIG>, the image processing apparatus <NUM> performs the 2D FFT twice by using the two cores <NUM> and <NUM>, and stores the depth-summed data in the memory <NUM> unlike in <FIG>.

Second, the first core <NUM> outputs the 1D FFT-transformed data to the memory <NUM>.

Third, the second core <NUM> reads data from the memory <NUM> in the column direction. The second core <NUM> performs the 1D FFT with respect to the read data. The second core <NUM> performs the 1D FFT in the column direction.

Sixth, the operators <NUM> output the depth-summed data to the memory <NUM>.

Seventh, the first core <NUM> reads data from the memory <NUM> in the row direction. The first core <NUM> performs the 1D FFT with respect to the read data.

Eighth, the first core <NUM> outputs the transformed data to the memory <NUM>.

Ninth, the second core <NUM> reads the data that is 1D-FFT transformed by the first core <NUM> from the memory <NUM>. The second core <NUM> performs the 1D FFT with respect to the read data. The first core <NUM> performs the 1D FFT in the column direction.

Ninth, the second core <NUM> outputs the 2D FFT-transformed data.

<FIG> is a timing example of data input to a first core according to an embodiment. The first core <NUM> processes data as shown in <FIG> to reduce a delay time. With reference to <FIG>, a description will be made of an example in which the first core <NUM> includes <NUM>1D FFT processors.

Reference numeral <NUM> indicates a sequence of data, and reference numeral <NUM> indicates a structure of detailed data in the sequence <NUM>. The <NUM>st FFT may be FFT from a pupil to a retina, and the <NUM>nd FFT may be FFT from a panel to the pupil. F[n] indicates an nth frame, and F[n-<NUM>] indicates an (n-<NUM>)th frame. L[<NUM>] indicates a <NUM>th line. Thus, F[n]L[<NUM>] indicates a <NUM>th line of an nth frame. When the first core <NUM> performs the 1D FFT in the row direction, L[<NUM>] indicates a <NUM>th row. L[<NUM>:<NUM>] indicates lines from the <NUM>th line to a <NUM>th line. Reference numerals <NUM>, <NUM>, and <NUM> indicate data processed during one time period, respectively. The nth line may include a plurality of lines according to a depth level. For example, if a depth level is <NUM>, L[<NUM>] may include <NUM> lines according to a depth.

The first core <NUM> processes the current frame and a previous frame by performing time division. That is, the first core <NUM> alternately performs the primary FFT with respect to the current frame and the secondary FFT with respect to the previous frame. For example, the first core <NUM> may perform the 1D FFT with respect to the nth frame during a <NUM>st time period to an <NUM>th time period, and perform the 1D FFT with respect to an (n-<NUM>)th frame during a <NUM>th time period. The 1D FFT with respect to the nth frame is the primary 1D FFT, and the 1D FFT with respect to the (n-<NUM>)th frame is the secondary 1D FFT.

During the <NUM>st through <NUM>th time periods (a total of <NUM> time periods), the first core <NUM> performs the primary 1D FFT with respect to a <NUM>th line <NUM> through a <NUM>th line <NUM> of the nth frame. If the first core <NUM> includes <NUM>1D FFT processors, the first core <NUM> performs the primary 1D FFT with respect to the <NUM>th line <NUM> of a <NUM>th depth through the <NUM>th line <NUM> of a <NUM>th depth during the <NUM>st time period. The first core <NUM> performs the 1D FFT with respect to <NUM> depths of one line during the <NUM>st time period. The first core <NUM> performs the primary 1D FFT with respect to a <NUM>st line of a <NUM>th depth through a <NUM>st line of a <NUM>th depth during the <NUM>nd time period. The first core <NUM> performs the primary 1D FFT with respect to a <NUM>th line <NUM> of the <NUM>th depth through a <NUM>th line <NUM> of the <NUM>th depth during the <NUM>th time period.

The first core <NUM> performs the primary 1D FFT with respect to a <NUM>th line <NUM> through a <NUM>th line <NUM> of of the (n-<NUM>)th frame during the <NUM>th time period. The <NUM>th line <NUM> through the <NUM>th line <NUM> are depth-summed lines. For example, the line <NUM> may be generatd by summing <NUM>th through <NUM>th depth data with respect to the <NUM>th line. D[<NUM>:<NUM>] indicates the depth-summed data.

As illustrated in <FIG>, the first core <NUM> processes data with respect to two frames through time division to reduce a time between the completion of Fourier transform with respect to the (n-<NUM>)th frame and the completion of Fourier transform with respect to the nth frame.

<FIG> is a timing example of data input to a secpnd core according to an embodiment. The second core <NUM> processes data as shown in <FIG> to reduce a time needed to store and read the data in and from the memory <NUM>. The second core <NUM> alternately performs the primary FFT with respect to the current frame and the secondary FFT with respect to the current frame. With reference to <FIG>, a description will be made of an example in which the second core <NUM> includes <NUM>1D FFT processors.

Reference numeral <NUM> indicates a sequence of data, and reference numeral <NUM> indicates a structure of detailed data in the sequence <NUM>. The second core <NUM>, unlike the first core <NUM>, may perform the 1D FFT with respect to the current frame.

During the <NUM>st time period, the second core <NUM> processes lines <NUM> through <NUM> for a total of <NUM> depths. If the second core <NUM> includes <NUM>1D FFT processors, the second core <NUM> may simultaneously process a total of <NUM> lines during the <NUM>st time period.

The second core <NUM> performs the primary 1D FFT with respect to the <NUM>th line <NUM> of of the nth frame during the <NUM>st time period. The line <NUM> includes the lines <NUM> through <NUM> for the total of <NUM> depths. Thus, during the <NUM>st time period, the second core <NUM> performs the primary 1D FFT with respect to the <NUM> lines <NUM> through <NUM> at the same time.

The second core <NUM> performs the secondary 1D FFT with respect to the <NUM>th line <NUM> of of the nth frame during the <NUM>nd time period. The <NUM>th line <NUM> is depth-summed data and includes one line <NUM>. Thus, the line <NUM> is processed by the 1D FFT processor, and the other ID FFT processors are in an idle state.

After the transformed data is depth-summed during the <NUM>st time period, the depth-summed data is input to the second core <NUM> during the <NUM>nd time period, thereby omitting the process of storing and reading the data in and from the memory <NUM>. If the first core <NUM> performs the 1D FFT in the row direction and the second core <NUM> performs the 1D FFT in the column direction, data transformed by the second core <NUM> needs to be read in the row direction after being stored in the memory <NUM> so as to be input to the first core <NUM>, but the data transformed by the second core <NUM> is located in the same column and thus may be input to the second core <NUM> without being stored in the memory <NUM>.

<FIG> is a flowchart illustrating an image processing method according to an example embodiment. Referring to <FIG>, the image processing apparatus <NUM> may perform the primary 2D FFT and the secondary 2D FFT via the first core <NUM> and the second core <NUM>. The first core <NUM> may perform the 1D FFT in the row direction, and the second core <NUM> may perform the 1D FFT in the column direction. In another example embodiment, the first core <NUM> may perform the 1D FFT in the column direction, and the second core <NUM> may perform the 1D FFT in the row direction.

In operation <NUM>, the first core <NUM> and the second core <NUM> may perform the primary 2D FFT and the secondary 2D FFT with respect to image data. The first core <NUM> may perform the primary 1D FFT with respect to the image data, and the second core <NUM> may perform the primary 1D FFT with respect to the data transformed by the first core <NUM>.

The first core <NUM> may read the image data in the row direction and performs the 1D FFT with respect to each line read in the row direction. The data transformed by the first core <NUM> may be stored in the memory <NUM> and may be read by the controller <NUM> in the column direction. The controller <NUM> may input the read data to the second core <NUM>. The second core <NUM> may perform the 1D FFT with respect to each line read in the column direction.

In operation <NUM>, the image processing apparatus <NUM> may perform the focus term operation and the depth summation operation with respect to the primary 2D FFT-transformed data. The primary 2D FFT-transformed data is data in the column direction of the frame and may be output from the second core <NUM> and.

In operation <NUM>, the first core <NUM> and the second core <NUM> may perform the secondary 2D FFT with respect to the depth-summed data. The second core <NUM> may perform the 1D FFT with respect to the depth-summed data. The data output from the second core <NUM> may be stored in the memory <NUM>. The controller <NUM> may read the data output from the second core <NUM>, which is stored in the memory <NUM>. The controller <NUM> may input the read data in the first core <NUM>. The first core <NUM> may perform the 1D FFT with respect to the input data.

According to the present disclosure, a Fourier transform may be performed by using a small number of cores.

A data processing sequence may be controlled to prevent a data processing process after the Fourier transform from being delayed.

An apparatus according to the present disclosure may a processor, a memory for storing program data and executing it, a permanent storage such as a disk drive, a communications port for communicating with external devices, and user interface devices, such as a touch panel, a key, a button, etc. Methods implemented with a software module or algorithm may be stored as computer-readable codes or program instructions executable on the processor on computer-readable recording media. Examples of the computer-readable recording media may include a magnetic storage medium (e.g., read-only memory (ROM), random-access memory (RAM), floppy disk, hard disk, etc.) and an optical medium (e.g., a compact disc-ROM (CD-ROM), a ditial versatile disc (DVD), etc.) , and so forth. The computer-readable recording medium may be distributed over network coupled computer systems so that a computer-readable code is stored and executed in a distributed fashion. The medium may be read by a computer, stored in a memory, and executed by a processor.

The present disclosure may be represented by block components and various process operations. Such functional blocks may be implemented by various numbers of hardware and/or software components which perform specific functions. For example, the present disclosure may employ various integrated circuit components, e.g., memory elements, processing elements, logic elements, look-up tables, and the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices. Similarly, where the elements of the present disclosure are implemented using software programming or software elements the disclosure may be implemented with any programming or scripting language such as C, C++, Java, assembler, or the like, with the various algorithms being implemented with any combination of data structures, objects, processes, routines or other programming elements. Functional aspects may be implemented as an algorithm executed in one or more processors. Furthermore, the present disclosure may employ any number of conventional techniques for electronics configuration, signal processing and/or control, data processing and the like. The term "mechanism", "element", "means", or "component" is used broadly and is not limited to mechanical or physical embodiments. The term may include a series of routines of software in conjuction with the processor or the like.

Particular executions described in the current embodiments are merely examples, and do not limit a technical range with any method. For the sake of brevity, conventional electronics, control systems, software development and other functional aspects of the systems may not be described in detail. Furthermore, the connecting lines, or connectors shown in the various figures presented are intended to represent exemplary functional relationships and/or physical or logical couplings between the various elements.

In the present disclosure (especially, in the claims), the use of "the" and other demonstratives similar thereto may correspond to both the singular form and the pluaral form. Also, if a range is described in the present disclosure, the range has to be regarded as including inventions adopting any individual element within the range (unless described otherwise), and also, it has to be regarded as having written in the detailed description of the dislosure each individual element included in the range. Unless the order of operations of a method is explicitly mentioned or described otherwise, the operations may be performed in a proper order. The order of the operations is not limited to the order the operations are mentioned. The use of all examples or exemplary terms (e.g., "etc.,", "and (or) the like", and "and so forth") is merely intended for descriptive purposes, and the scope is not necessarily limited by the examples or exemplary terms unless defined by the claims. Also, one of ordinary skill in the art may appreciate that the present disclosure may be configured through various modifications, combinations, and changes according to design conditions and factors without departing from the technical scope of the present disclosure and its equivalents.

It should be understood that the example embodiments described herein should be considered in a descriptive sense only and not for purposes of limitation. Descriptions of features or aspects within each example embodiment should typically be considered as available for other similar features or aspects in other embodiments.

Claim 1:
An image processing apparatus (<NUM>) configured to perform a two-dimensional, 2D, fast Fourier transform, FFT, with respect to image data, the image processing apparatus comprising:
a first core (<NUM>) and a second core (<NUM>), each of the first core and the second core comprising a plurality of processors configured to perform a one-dimensional, 1D, FFT;
a memory (<NUM>) configured to store data output from the first core and the second core; and
a controller (<NUM>) configured to control the first core and the second core to perform a primary 2D FFT and a secondary 2D FFT with respect to the image data by repeatedly performing the 1D FFT, wherein the image processing apparatus further comprises:
a plurality of operators configured to generate depth-summed image data by arranging primary 2D FFT-transformed data generated by the first core and the second core into a plurality of lines having a predetermined depth level and sequentially summing a subset of the plurality of lines, wherein a number of lines included in the subset is equal to the predetermined depth level,
wherein the controller is configured to store the depth-summed image data in the memory, read the depth-summed data from the memory, and input the read data to the first core and wherein the controller is further configured to input the depth-summed image data to the second core, thereby omitting the process of storing the depth-summed image data in and reading the depth-summed image data from the memory when the depth-summed image data is input to the second core.