Patent Description:
Image acquisition devices project a three-dimensional scene onto a two-dimensional sensor. During operation, a conventional capture device captures a two-dimensional (<NUM>-D) image of the scene representing an amount of light that reaches a photosensor (or photodetector or photosite) within the device. However, this <NUM>-D image contains no information about the directional distribution of the light rays that reach the photosensor, which may be referred to as the light-field. Depth, for example, is lost during the acquisition. Thus, a conventional capture device does not store most of the information about the light distribution from the scene.

Light-field capture devices, also referred to as "light-field data acquisition devices", have been designed to measure a four-dimensional (4D) light-field of the scene by capturing the light from different viewpoints of that scene. Thus, by measuring the amount of light traveling along each beam of light that intersects the photosensor, these devices can capture additional optical information, in particular information about the directional distribution of the bundle of light rays, for providing new imaging applications by post-processing. The information acquired/obtained by a light-field capture device is referred to as the light-field data. Light-field capture devices are defined herein as any devices that are capable of capturing light-field data. There are several types of light-field capture devices, among which:.

Light-field data processing comprises notably, but is not limited to, generating refocused images of a scene, generating perspective views of a scene, generating depth maps of a scene, generating extended depth of field (EDOF) images, generating stereoscopic images, and/or any combination of these.

Images or videos acquired by light-field acquisition devices may need to be transmitted to other devices, for example to display devices.

When the user wants to display the light-field content on its display device, he/she may select different portions of the content to bring different portions of the content into focus and out of focus. The focus plane is set at the desired depth, and the depth data is used to refocus the portions of the image selected by the user.

However, the picture synthesized accordingly may lack realism and aesthetic quality. Actually, out-of-focus parts of an image produced by a lens are blurred. The aesthetic quality of such a blur is called bokeh, which may be defined as "the way the lens renders out-of-focus points of light". Differences in lens aberrations and aperture shape cause some lens designs to blur the image in a way that is pleasing to the eye, while others produce blurring that is unpleasant or distracting.

When rendering an image or a video from a light-field content, it would be interesting to display an image as close as possible as a conventional image or video in terms of blur aesthetics, i.e. to display an image with good bokeh properties.

It would hence be desirable to provide a technique for encoding and decoding a signal representative of a light-field content, which would be appropriate to the specificities of light-field imaging, and which would allow realistic and/or aesthetic rendering of image or video contents.

In this regard, document <CIT> discloses a method for enabling a user of a display screen device to experience an enhanced spatial perception of plenoptic content. The method comprises tracking a gaze fixation of a user and timing over a set interval an accumulated time in which the user's gaze fixation rests on each of a plurality of at least two depth planes which have been associated with a plurality of at least one plenoptic image, and refocusing a display relating to the sequence at least one plenoptic image. In order to enhance spatial perception by the user, bokeh may be artificially increased.

Furthermore, document <CIT> discloses a system for the manipulation of captured light field image data. A depth map includes depth information for one or more pixels in the image data. The system includes a processor, a display, a user input device, and a memory. An image manipulation application configures the processor to display a first synthesized image, receive user input data identifying a region within the first synthesized image, determine boundary data for the identified region using the depth map, receive user input data identifying at least one action, and perform the received action using the boundary data and the captured light field image data. The received action may be a bokeh modification action.

The present invention has been devised with the foregoing in mind.

The invention is as defined by the appended independent claims. The dependent claims include advantageous further developments and improvements of the present principles as described below.

According to a first aspect of the invention there is provided a computer implemented method for generating data representative of a bokeh associated with a volume in an object space of an optical system for acquiring light-field content occupied by a set of rays of light passing through a pupil of said optical system and a conjugate, in said object space of said optical system, of at least one pixel of a sensor associated with said optical system, said volume occupied by said set of rays of light being called a pixel beam, from a collection of such pixel beams representing a light-field content, said method comprising for at least one pixel beam of said collection of pixel beams:.

Such a method enables to convey a synthetic bokeh shape associated with a given pixel beam. An advantage of such a method is that it preserves the geometry of the underlying physics. The method according to an embodiment of the invention is fast and presents the advantage of intrinsically conveying a bokeh shape throughout an imaging workflow of a light-field content represented by a collection of pixel beams.

Another advantage of the method according to an embodiment of the invention is that such a method is generic since it relies on the use of polygons.

According to the invention, the section of the pixel beam corresponds to the conjugate of a pixel of the sensor, the pupil corresponds to an entrance pupil of the optical system and the first surface corresponds to a focus plane of said optical system.

This corresponds to synthetic imaging. The pixel beam is an object pixel beam which is located in the object space of the optical system.

According to an embodiment of the method for generating data representative of a bokeh, the number of rays joining the p vertices of the first polygon to the n vertices of the second polygon is a multiple of the product of n by p.

It appears that the complexity of the solution according to an embodiment of the invention increases linearly with the numbers of vertices of the first and the second polygons. Multiples of four are preferred for p and n for alignment issues in memory units during processing.

In the case of pixel beams located in the object space of the optical system, the bokeh obtained according to an embodiment of the invention corresponds to the convex envelope of p multiplied by n points.

Another object of the invention is an apparatus for generating data representative of a bokeh associated with a volume in an object space of an optical system for acquiring light-field content occupied by a set of rays of light passing through a pupil of said optical system and a conjugate, in said object space of said optical system, of at least one pixel of a sensor associated with said optical system, said volume occupied by said set of rays of light being called a pixel beam, from a collection of such pixel beams representing a light-field content, said apparatus comprising a processor configured to, for at least one pixel beam of said collection of pixel beams:.

According to an embodiment of the apparatus for generating data representative of a bokeh, the number of rays joining the p vertices of the first polygon to the n vertices of the second polygon is a multiple of the product of n by p.

Some processes implemented by elements of the invention may be computer implemented. Accordingly, such elements may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system'. Furthermore, such elements may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

Since elements of the present invention can be implemented in software, the present invention can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RF signal.

As will be appreciated by one skilled in the art, aspects of the present principles can be embodied as a system, method or computer readable medium. Accordingly, aspects of the present principles can take the form of an entirely hardware embodiment, an entirely software embodiment, (including firmware, resident software, micro-code, and so forth) or an embodiment combining software and hardware aspects that can all generally be referred to herein as a "circuit", "module", or "system". Furthermore, aspects of the present principles can take the form of a computer readable storage medium. Any combination of one or more computer readable storage medium(a) may be utilized.

For any optical acquisition system, may it be plenoptic or not, in addition to raw images or epipolar images representing 4D light-field data captured by the optical acquisition system, it is interesting to obtain information related to a correspondence between pixels of a sensor of said optical acquisition system and an object space of said optical acquisition system. Knowing which portion of the object space of an optical acquisition system a pixel belonging to the sensor of said optical acquisition system is sensing enables the improvement of signal processing operations such as de-multiplexing, de-mosaicking, refocusing, etc., and the mixing of images captured by different optical systems with different characteristics. Furthermore, information related to the correspondence between the pixels of the sensor of the optical acquisition system and the object space of said optical acquisition system are independent of the optical acquisition system.

The present disclosure introduces the notion of pixel beam <NUM>, shown on <FIG>, which represents a volume occupied by a set of rays of light passing through a pupil of an optical system <NUM> of a camera (not shown on <FIG>) and a conjugate of a pixel of a sensor of the camera in an object space of the optical system in a direction normal to a surface of the pupil
The set of rays of light is sensed by a pixel <NUM> of a sensor <NUM> of the camera through a pupil <NUM> of said optical system <NUM>. The optical system <NUM> may be a combination of lenses fit for photo or video cameras. A pupil of an optical system is defined as the image of an aperture stop as seen through said optical system, i.e. the lenses of the optical acquisition system, which precedes said aperture stop. An aperture stop is an opening which limits the amount of light which passes through the optical system of the optical acquisition system. For example, an adjustable blade diaphragm located inside a camera lens is the aperture stop for the lens. The amount of light admitted through the diaphragm is controlled by the diameter of the diaphragm opening which may adapted depending of the amount of light a user of the camera wishes to admit. For example, making the aperture smaller reduces the amount of light admitted through the diaphragm, and, simultaneously, increases the depth of focus. The apparent size of a stop may be larger or smaller than its physical size because of the refractive action of a portion of the lens. Formally, a pupil is the image of the aperture stop through all lenses of the optical acquisition system located between the physical stop and the observation space.

A pixel beam <NUM> is defined as a pencil of rays of light that reach a given pixel <NUM> when propagating through the optical system <NUM> via an entrance pupil <NUM>. As light travels on straight lines in free space, the shape of such a pixel beam <NUM> can be defined by two sections, one being the conjugate <NUM> of the pixel <NUM>, and the other being the entrance pupil <NUM>. The pixel <NUM> is defined by its non-null surface and its sensitivity map.

In a first embodiment of the invention, a pixel beam <NUM> may be represented by an hyperboloid of one sheet, as shown on <FIG>, supported by two elements: the pupil <NUM> and the conjugate <NUM> of the pixel <NUM> in the object space.

A hyperboloid of one sheet is a ruled surface that can support the notion of pencil of rays of light and is compatible with the notion of "étendue" of physical light beams, notion linked to the preservation of energy across sections of the physical light beams.

As represented on <FIG>, a hyperboloid of one sheet <NUM> is mostly identical to its asymptotic cones <NUM>, <NUM>, except in the fundamental region of its smallest section, called the waist <NUM>, which corresponds to the conjugate <NUM> in the object space. For plenoptic systems, such as light-field cameras, this is the region where space sampling by multiple path rays is performed. Sampling space with unique cones degenerating to a point in this region is not adequate, as pixel <NUM> sensitivity is significant on some tens of square microns on its surface and cannot be represented by a mathematical point with infinitely small surface as would be a cone tip.

In an embodiment of the invention, each pixel beam <NUM>, <NUM>, <NUM>, is defined by four independent parameters: zP, θx, θy, a defining the position and size of the pixel conjugate <NUM>, <NUM>, in front of the pupil <NUM>, <NUM> and by six pupilar parameters xO, yO, zO, θx<NUM>, θy<NUM>, r which define the position, orientation and radius of the pupil <NUM>, <NUM>. These six pupilar parameters are common to the collection of pixel beams, when represented by a hyperboloid of one sheet, sharing a same pupil <NUM>, <NUM>. Indeed, a pixel beam represents the volume occupied by a set of rays of light in the object space of the optical system <NUM> sensed by the pixel <NUM> through the pupil <NUM>, i.e. to a given couple pixel <NUM>/pupil <NUM>, <NUM> corresponds a unique pixel beam <NUM>, <NUM>, <NUM>, but a plurality of distinct pixel beams can be supported by a same pupil <NUM>, <NUM>.

An origin O of a coordinate system (x, y, z) in which the parameters of the hyperboloid of one sheet representing the pixel beam <NUM>, <NUM>, <NUM> are defined corresponds to the centre of the pupil <NUM> as shown on <FIG>, where the z axis defines a direction normal to the surface of the pupil <NUM>, <NUM>.

The parameters θx, θy, define chief ray directions relative to the entrance of the pupil <NUM> centre. They depend on the pixel <NUM> position on the sensor <NUM> and on the optical elements of the optical system <NUM>. More precisely, the parameters θx, θy represent shear angles defining a direction of the conjugate <NUM> of the pixel <NUM> from the centre of the pupil <NUM>.

The parameter ZP represents a distance of the waist <NUM> of the pixel beam <NUM>, <NUM>, <NUM>, or the conjugate <NUM> of the pixel <NUM>, along the z axis.

The parameter a represents the radius of the waist <NUM> of the pixel beam <NUM>, <NUM>, <NUM>.

For optical systems <NUM> where optical distortions and field curvatures may be modelled, the parameters ZP and a can depend on the parameters θx and θy via parametric functions.

The four independent parameters are related to the pixel <NUM> and its conjugate <NUM>.

The six complementary pupilar parameters defining a pixel beam <NUM>, <NUM>, <NUM> are:.

These six pupilar parameters are related to the pupil <NUM>, <NUM>. Another parameter c is defined. Such a parameter c is dependent on the parameters zP and a related to the pixel <NUM> and its conjugate <NUM> and on the parameters r related to the pupil <NUM>, <NUM>. The parameter c defines the angular aperture α of the pixel beam <NUM>, <NUM>, <NUM> and is given by the formula: <MAT>.

Thus, the expression of the parameter c is given by the following equation: <MAT>.

The coordinates (x, y, z), in the object space, of points belonging to the surface delimiting the pixel beam <NUM>, <NUM>, <NUM> are function of the above defined sets of parameters related to the pupil <NUM>, and to the conjugate <NUM> of the pixel. Thus, equation (<NUM>) enabling the generation of the hyperboloid of one sheet representing the pixel beam <NUM>, <NUM>, <NUM> is: <MAT>.

A parametric equation (<NUM>) of the same hyperboloid representing the pixel beam <NUM>, <NUM>, <NUM> is: <MAT> wherein v is an angle in the (x, y) plane enabling the generation of the pixel beam <NUM>, <NUM>, <NUM> from a generating hyperbola, v varies in [<NUM>, <NUM>π] interval, and z ∈ [<NUM>, ∞] is the coordinate along the z axis which defines a direction normal to the surface of the pupil <NUM>, <NUM>. Equations (<NUM>) and (<NUM>) are written on the assumption that the section of the pixel <NUM> and its conjugate <NUM> are circular and that the section of the pupil <NUM>, <NUM> is circular as well.

Information related to a correspondence between pixels of a sensor of said optical acquisition system and an object space of said optical acquisition system may take the form of either a set of parameters comprising the four independent parameters: zP, θx, θy, a defining the position and size of the pixel conjugate <NUM>, <NUM>, in front of the pupil <NUM>, <NUM> and the six pupilar parameters xO, yO, zO, θx<NUM>, θy<NUM>, r which define the position, orientation and radius of the pupil <NUM>, <NUM> when the pixel beam is to be represented by its parametric equation.

Thus, this set of parameters is provided in addition to raw images or epipolar images representing 4D light-field data captured by the optical acquisition system in order to be used while processing the 4D light-field data.

In a second embodiment of the invention, a pixel beam <NUM> may be represented by two coaxial, partially overlapping cones a front cone <NUM>F and a rear cone <NUM>R as shown on <FIG>, supported by two elements: the pupil <NUM> and the conjugate <NUM> of the pixel <NUM> in the object space, i.e. the surface in the object space that is imaged on the pixel.

The front cone <NUM>F is the image of a convex frustum defined by the pixel <NUM> and the pupil <NUM>. The apex of the convex frustum lies beyond the sensor of the optical acquisition system. By construction, the front cone <NUM>F is converging in the object space of the optical acquisition system and the apex of the front cone <NUM>F lies between the conjugate of the pixel <NUM>, or the waist of the pixel beam <NUM>, and the pupil <NUM>. The front cone <NUM>F derives from the solid angle subtended by the pupil <NUM> at the pixel <NUM>.

The rear cone <NUM>R is the image of a cone defined by the pixel <NUM> and the pupil <NUM>, the apex of which lies between the pupil <NUM> and the sensor of the optical acquisition system. By construction, the apex of the rear cone <NUM>R is located beyond the waist <NUM> of the pupil <NUM>. The rear cone <NUM>R does not necessarily converge in the object space of the optical acquisition system, in some cases it may degenerate into a cylinder or a diverging cone. In the latter case, the apex of the diverging cone lies in the image space of the optical acquisition system, i.e. before the entrance of the pupil <NUM>.

The front cone <NUM>F and the rear cone <NUM>R share the same revolution axis, which is a line joining the centre of the pupil <NUM> and the centre of the waist <NUM>.

Cones are ruled surfaces that can support the notion of pencil of rays of light and when combining two cones is compatible with the notion of "étendue" of physical light beams, notion linked to the preservation of energy across sections of the physical light beams. Intersections of cones with planes are conic curves, as for hyperboloids, which can be characterized by a plurality of coefficients. Considering its apex, a cone may be represented by three angular parameters: a polar angle measured from the revolution axis of the cone, up to the apex angle and the direction of the revolution axis given by two angles.

Let xyz be the coordinate system of the optical acquisition system, z denoting the optical axis of the optical acquisition system with z > <NUM> in the object space of the optical acquisition system and the centre of the pupil <NUM> being the origin of said coordinate system.

The optics of the optical acquisition system images the object space of the optical acquisition system from the range z ∈ [<NUM>f; +∞] into the image space of the optical acquisition system z ∈ [-<NUM>f; -f], where f is the focal length of the optics of the optical acquisition system. The location of the pupil <NUM> and the waist <NUM> of the pixel beam <NUM> are known in the coordinate system xyz of the optical acquisition system from the calibration of the optical acquisition system. The pupil <NUM> and the waist <NUM> are assumed to be parallel and are both normal to the z axis.

Let us call z' the chief ray of the pixel beam <NUM>. The chief ray is the line joining the centre of the pupil <NUM> and the centre of the waist <NUM> of the pixel beam <NUM>. The chief ray is also the revolution axis and the axis of symmetry of the pixel beam <NUM>. Thus, in the coordinate system xyz', the pixel beam <NUM> is a solid of revolution.

Both the apices of the front cone <NUM>F and the rear cone <NUM>R are located on the chief ray z' of the pixel beam <NUM>. Under the thin lens approximation, the coordinates of these two apices are computed in the coordinate system xyz of the optical acquisition system as follow, under the assumption that the sensor of the optical acquisition system is not located the rear focal plane: <MAT> i.e.: <MAT> where P, zP, W and zw respectively denote the diameter of the pupil <NUM> with P > <NUM>, its z-coordinate, the diameter of the pixel's conjugate <NUM> with <NUM> < W < +∞, and its z-coordinate <NUM> < zw < +∞.

The z-coordinate zrear of the apex of the rear cone <NUM>R may be positive, when the rear cone <NUM>R is a converging cone, negative, when the rear cone <NUM>R is a diverging cone. It may also be infinite if the pupil <NUM> and the pixel's conjugate <NUM> of the pixel beam are of the same size.

If the sensor of the optical acquisition system is located on the rear focal plane, then W = +∞ and zw = +∞. As their ratio is a constant: <MAT> where p and f respectively represent the diameter of the pixel <NUM> with p > <NUM> and the focal length of the optics of the optical acquisition system with f > <NUM> assuming the optics of the optical acquisition system is a converging lens.

Considering the apex of each cones, which union represents a pixel beam <NUM>, rays can be defined with two angular parameters: the polar angle measure from the revolution axis of the pixel beam, up to the apex angle, and an azimuth in [<NUM>,<NUM>π [.

Those information related to pixel beams are metadata associated with a given optical acquisition system. They may be provided as a data file stored for example on a CD-ROM or a flash drive supplied with the optical acquisition system. The data file containing the additional information related to pixel beams may also be downloaded from a server belonging to the manufacturer of the optical acquisition system. In an embodiment of the invention, these additional information related to pixel beams may also be embedded in a header of the images captured by the optical acquisition system.

The knowledge of these information related to pixel beams enables the processing of images captured by any optical acquisition system independently of the proprietary file format and of the features of the optical acquisition system used to capture the images to be processed.

The knowledge of information related to pixel beams enables the processing of images captured by any optical acquisition system independently of the proprietary file format and of the features of the optical acquisition system used to capture the images to be processed.

<FIG> is a schematic block diagram illustrating an example of an apparatus for generating data representative of a bokeh associated with a pixel beam according to an embodiment of the present disclosure.

The apparatus <NUM> comprises a processor <NUM>, a storage unit <NUM>, an input device <NUM>, a display device <NUM>, and an interface unit <NUM> which are connected by a bus <NUM>. Of course, constituent elements of the computer apparatus <NUM> may be connected by a connection other than a bus connection.

The processor <NUM> controls operations of the apparatus <NUM>. The storage unit <NUM> stores at least one program capable of generating data representative of pixel beams representing the object space of a first optical system when these pixel beams are imaged through a second optical system to be executed by the processor <NUM>, and various data, including parameters related to a position of the pixel <NUM> on the sensor <NUM> or parameters related to the first optical system <NUM> of the optical acquisition system and a second optical system, parameters used by computations performed by the processor <NUM>, intermediate data of computations performed by the processor <NUM>, and so on. The processor <NUM> may be formed by any known and suitable hardware, or software, or a combination of hardware and software. For example, the processor <NUM> may be formed by dedicated hardware such as a processing circuit, or by a programmable processing unit such as a CPU (Central Processing Unit) that executes a program stored in a memory thereof.

The storage unit <NUM> may be formed by any suitable storage or means capable of storing the program, data, or the like in a computer-readable manner. Examples of the storage unit <NUM> include non-transitory computer-readable storage media such as semiconductor memory devices, and magnetic, optical, or magneto-optical recording media loaded into a read and write unit. The program causes the processor <NUM> to perform a process for computing data representative of the pixel beams of a collection of pixel beams representative of the object space of a first optical system from an image conjugate of said pixel beam through a second optical system beam according to an embodiment of the present disclosure as described hereinafter with reference to <FIG>.

The input device <NUM> may be formed by a keyboard, a pointing device such as a mouse, or the like for use by the user to input commands, to make user's selections of parameters used for generating a parametric representation of a volume occupied by a set of rays of light in an object space of an optical system. The output device <NUM> may be formed by a display device to display, for example, a Graphical User Interface (GUI), images generated according to an embodiment of the present disclosure. The input device <NUM> and the output device <NUM> may be formed integrally by a touchscreen panel, for example.

The interface unit <NUM> provides an interface between the apparatus <NUM> and an external apparatus. The interface unit <NUM> may be communicable with the external apparatus via cable or wireless communication. In an embodiment, the external apparatus may be an optical acquisition system such as an actual camera.

<FIG> is a flow chart for explaining a process for generating data representative of a bokeh associated with a pixel beam according to an embodiment of the invention.

Out-of-focus parts of an image produced by a lens are blurred. The aesthetic quality of such a blur is called a bokeh, which may be defined as "the way the lens renders out-of-focus points of light". Differences in lens aberrations and aperture shape cause some lens designs to blur the image in a way that is pleasing to the eye, while others produce blurring that is unpleasant or distracting.

A light-field content may be sampled by a collection of pixel beams defining the object space of an optical system. Embodiments of the invention are not limited to light-field contents directly acquired by an optical device. These contents may be Computer Graphics Image (CGI) that are totally or partially simulated by a computer for a given scene description. Another source of light-field contents may be post-produced data that are modified, for instance color graded, light-field contents obtained from an optical device or CGI. It is also now common in the movie industry to have data that are a mix of both data acquired using an optical acquisition device, and CGI data. It is to be understood that the pixels of a sensor can be simulated by a computer-generated scene system and, by extension, the whole sensor can be simulated by said system. From here, it is understood that any reference to a "pixel of a sensor" or a "sensor" can be either a physical object attached to an optical acquisition device or a simulated entity obtained by a computer-generated scene system.

In a step <NUM>, the processor <NUM> of the apparatus <NUM> samples a section S1 of a pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> of the collection of pixel beams representing a light-field content as a first polygon P1 having p vertices. The section S1 may be a circle, an ellipse, a square, etc. The section S1 and examples of polygon P1 are represented on <FIG>. On <FIG>, the polygon P1 sampling the section S1 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> has <NUM> vertices. On <FIG>, the polygon P1 sampling the section S1 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> has <NUM> vertices. On <FIG>, the polygon P1 sampling the section S1 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> has <NUM> vertices and can almost merge with the section S1 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM>.

In a step <NUM>, the processor <NUM> of the apparatus <NUM> samples a pupil S2 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> of the collection of pixel beams representing a light-field content as a second polygon P2 having n vertices. The section S1 may be a circle, an ellipse, a square, etc. The pupil S2 and examples of polygon P2 are represented on <FIG>. On <FIG>, the polygon P2 sampling pupil S2 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> has <NUM> vertices. On <FIG>, the polygon P2 sampling pupil S2 of the pixel beam <NUM>, <NUM>, <NUM>,<NUM>,<NUM> has <NUM> vertices. On <FIG>, the polygon P2 sampling pupil S2 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> has <NUM> vertices. On <FIG>, the polygon P2 sampling pupil S2 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> has <NUM> vertices. On <FIG>, the polygon P2 sampling pupil S2 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM> has <NUM> vertices. On <FIG>, the polygon P2 sampling pupil S2 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> has <NUM> vertices. On <FIG>, the polygon P2 sampling pupil S2 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> has <NUM> vertices and can almost merge with the pupil S2 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM>.

It appears that the complexity of the method according to an embodiment of the invention increases linearly with the numbers of vertices of the first polygon P1 and the second polygon P2. Multiples of four are preferred for p and n for alignment issues in the storage unit <NUM> and the processor <NUM> of the apparatus <NUM> during processing. The number of vertices of the polygons P1 and P2 may be chosen by a user of the apparatus <NUM> and provided to said apparatus <NUM> as parameters to be used to generate data representative of the bokeh through the input device <NUM> of the apparatus <NUM>.

In a first embodiment of the method for generating data representative of a bokeh associated with a pixel beam, the considered pixel beams <NUM>, <NUM>, <NUM>, <NUM> belong to a collection of pixel beams representative of the object space of an optical system (not shown on the figures). In this first embodiment, the section S1 of the pixel beam <NUM>, <NUM>, <NUM>, <NUM> corresponds to the conjugate of the pixel <NUM>, <NUM>, <NUM>, <NUM>; the pupil S2 is an entrance pupil and corresponds to the pupil <NUM>, <NUM> and <NUM>.

In another variant of the method for generating data representative of a bokeh associated with a pixel beam, the considered pixel beams <NUM> are conjugates of pixel beams <NUM>, <NUM>, <NUM>, <NUM> belonging to a collection of pixel beams representative of the object space of a first optical system (not shown on the figures) through a second optical system <NUM> as represented on <FIG>. In the following example, the pixel beams <NUM>, <NUM>, <NUM>, <NUM>, <NUM> are represented by a hyperboloid of one sheet. In this second embodiment, the section S1 of the pixel beam <NUM> corresponds to a conjugate, through the second optical system <NUM>, of an intersection of a given pixel beam <NUM>, <NUM>, <NUM>, <NUM> with a plane; the pupil S2 corresponds to an exit pupil of the second optical system <NUM>.

Back top <FIG>, in a step <NUM>, the processor <NUM> of the apparatus <NUM> computes sets of rays joining together the p vertices of the first polygon P1 and the n vertices of the second polygon P2.

In the case of pixel beams <NUM>, <NUM>, <NUM>, <NUM> located in the object space of the optical system, which corresponds to the first embodiment of the method for generating data representative of a bokeh associated with a pixel beam, there are p multiplied by n rays joining the vertices of the first polygon P1 with the vertices of the second polygon P2.

In the case of pixel beams <NUM>, or image pixel beams; which corresponds to the further variant of the method for generating data representative of a bokeh associated with a pixel beam, and more specifically in the case where the pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM> corresponds to a union of two overlapping cones, there are n multiplied by 2p rays joining the vertices of the first polygon P1 with the vertices of the second polygon P2.

In a step <NUM>, the processor <NUM> computes intersections of the rays joining together the p vertices of the first polygon P1 and the n vertices of the second polygon P2 with a surface.

In the first embodiment of the method for generating data representative of a bokeh associated with a pixel beam, the surface corresponds to a focus plane of the optical system.

In the further variant of the method for generating data representative of a bokeh associated to a pixel beam, the surface corresponds to the sensor associated with the second optical system <NUM>.

More generally, the surface may correspond either to a focus plane when a computation of a section of a pixel beam is at stake, or to the sensor when imaging a section of a pixel beam in a camera.

In a step <NUM>, the processor <NUM> of the apparatus <NUM> computes a convex envelope fitting the intersections of the rays joining together the p vertices of the first polygon P1 and the n vertices of the second polygon P2 with the surface. The convex envelope corresponds to the bokeh associated with a pixel beam <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. The parameters representative of the convex envelope computed during step <NUM> may be stored in the storing unit <NUM> of the apparatus <NUM> for further uses such as rendering the light-field content sampled by the collection of pixel beams for which data related to bokeh have been computed. The more intersections the more aesthetic the bokeh.

Although the present invention has been described hereinabove with reference to specific embodiments, the present invention is not limited to the specific embodiments, and modifications will be apparent to a skilled person in the art which lie within the scope of the present invention.

Claim 1:
A computer implemented method comprising:
generating 4D light field data representative of at least one volume in an object space of an optical system (<NUM>) occupied by a set of rays of light passing through a pupil (<NUM>) of said optical system (<NUM>) and called a pixel beam (<NUM>) and through a conjugate (<NUM>) defining one section of said pixel beam (<NUM>), in said object space of said optical system (<NUM>), of at least one pixel (<NUM>) of a sensor (<NUM>) associated with said optical system (<NUM>), said conjugate of the at least one pixel being in a direction normal to a surface of the pupil (<NUM>), said pupil (<NUM>) being defined as an image of an aperture stop as seen through said optical system (<NUM>) and defining another one section of said pixel beam (<NUM>), characterized in that it further comprises
- sampling (<NUM>) a section (S1) of said pixel beam (<NUM>) as a first polygon (P1) having p vertices, said section (S1) corresponding to said conjugate (<NUM>) of said pixel (<NUM>) of said sensor (<NUM>);
- sampling (<NUM>) a pupil (S2) of said pixel beam (<NUM>) as a second polygon (P2) having n vertices, said pupil (S2) corresponding to an entrance pupil (<NUM>) of said optical system (<NUM>);
- computing (<NUM>) sets of rays joining together said p vertices of said first polygon (P1) and said n vertices of said second polygon (P2);
- computing (<NUM>) intersections of said rays joining together the p vertices of the first polygon (P1) and the n vertices of the second polygon (P2) with a surface;
- computing (<NUM>) a convex envelope fitting said intersections of the rays joining together the p vertices of the first polygon (P1) and the n vertices of the second polygon (P2) with the surface, said convex envelope corresponding to a bokeh associated with a pixel beam; and
- providing a set of parameters defining a correspondence between pixels and an object space of said optical system and defining each pixel beam (<NUM>) and representative of said convex envelope as 4D light field data representative of the bokeh associated to said pixel beam (<NUM>).