Image Based Lighting with Near Light Sources
Shiho Furuya, Takayuki ItohGraduate School of Humanitics and Sciences, Ochanomizu University
E-mail: {shiho, itot}@itolab.is.ocha.ac.jp

Abstract

realize natural illumination by using photographs as light

Recent some researches on realistic computer graphics

sources [1]. IBL uses an omni-directional photograph(CG) focus on synthesis of photographs and virtualtaken at the position where virtual objects locate, as theobjects. Image Based Lighting (IBL) is one of thelight source for global illumination. Designers do nottechniques to generate naturally illuminated images byneed to finely set up virtual light sources in IBL.

using photographs. This paper reports a new experiment

Therefore, even non-expert designers can create realistic

to render virtual objects into real scenes with near lightnaturally illuminated scenes by applying IBL.

sources, which have not been considered in original IBL.

However, existing IBL techniques usually assume

Our experiment shows that we can generate more

light sources infinitely far away from virtual objects.

realistic images by faithfully constructing near light

This is one of the problems of IBL, because illumination

source environment.

of the real world cannot be limited to distant light

1. Introduction

sources. We often have to set up light sources locating

We have a long history of research on realistic computernear virtual objects to generate realistic synthesizedgraphics (CG). We also have a long history of geometricimages, but original IBL does not suppose such situation.

modeling of artificial objects, such as automobile andWe think this problem is important in some use cases;

building, by using computer aided design (CAD) systems.

for example, let us suppose to use IBL for production ofGenerally we need high-level skill and a long time forpamphlets or promotional films in the manufacturing

very realistic CG image creation, if we manually designindustry. Especially we think IBL is useful to synthesize

all objects in a scene. Recent CG researchers haveunreleased new product designed using CAD systems

focused on synthesis of photographs and manually

onto real scenes. In this situation we can easily assume

designed artificial objects, as a reasonable technique tothe scenes including light sources near the position

generate more realistic images.

where we would like to put the virtual objects. We thinkLet us say the scene in the photograph as “real scene”,
and artificial objects to be synthesized to the photographit is important to reconstruct the near light sources in thephotograph to realistically illuminate the virtual object.

as “virtual object”. We should balance two kinds of light

This paper reports a new experiment to render virtualsources, where one is the virtual light source illuminatingobjects into real scenes, with near light sources, which

the virtual objects, and the other is the real light source

have not been considered in the original IBL. The

illuminating the real scene in the photograph. Originallytechnique presented in this paper reconstructs the near

design of light sources needs a lot of skill and time, andlight sources, which are in the real scene and nearby themoreover, the reality of CG image strongly depends onvirtual objects. It calculates illumination using thethe design of light sources, even when we do notreconstructed near light sources, in addition to thesynthesis with the photograph. Balance of two kinds ofomni-directional photograph as distant light sources,

light sources is a further difficult problem.

such as a fluorescent light on a ceiling of the room.

Image Based Lighting (IBL) is a novel technique to1

2. Related works

are shot in the local image and nearby virtual objects, it

The technique presented in this paper is an extension of

may be inconsistent between the calculation result of

IBL proposed by Debevec [1]. IBL requires following

global illumination and the lighting in the local image. It

three items to generate synthesized image:is possible to take a photograph for distant image so that•

Virtual objects are defined as three dimensional

it contains the near light sources, in order to solve this(3D) geometric models designed using CG or CAD

problem. However, we do not think it is the fundamentalsystems.

solution for this problem, because one reason of theLocal image is defined as a photograph taken at aninconsistency is difference of degree of incidence of theoptional viewpoint. IBL generates an image

illumination between distant and near light sources.

synthesizing the local image and the virtual objects.

Unless we set up independent near light sources inDistant image is defined as a photograph ofaddition to the distant image, we cannot consider of theomni-directional scene taken at the position wheredifference of irradiation effects between distant and nearvirtual objects locate.

light sources.

••

Figure 1 denotes the procedure of IBL. Here, IBL

Sato et al. also presented an IBL-like technique [2]assumes a floor, where virtual objects locate around thewhich determines light environments by shooting

origin of a three-dimensional coordinate system. It alsoomni-directional photographs of the real world using aassumes walls which surround virtual objects from far

fish-eye lens. The light environments measured from theaway, where a distant image is mapped on the walls. It

omni-directional photograph are approximated as threesupposes pixels of the distant image as light source, todimensional triangular elements, whose vertices are

illuminate the virtual objects. It finally synthesizes the

mainly locate at the portions those intensities are high.

local image and the virtual objects considering of theThe technique then calculates the illumination and shade

light distribution computed by the above process.

of virtual objects using the light environments, andfinally synthesizes them to a photograph. The technique1)

distant2) Illuminating relatively theimage on the wallbrightness of pixels on theMapping

is applicable for both indoor and outdoor scenes;
however, it does not import near light sources.

wall

Okabe et al presented an image analysis techniqueunder near light sources, and an image composition
technique as an application of the analysis [3]. However,
the technique requires several images under various
lighting conditions.

3)

3. Rendering
scenes
3.1 Overview

Synthesizing local
image and virtualobjects

virtualobjects

into

real

Similar to the original IBL, the technique presented inthis paper also requires three items, virtual objects, aFigure 1: Procedure of IBL.

local image, and a distant image, as explained in Section
IBL simplifies the process of lighting design for global

2. The technique realistically synthesizes the virtual

illumination, because the only we need to prepare is anobjects into the real scene, by using the three items andomni-directional photograph. However, the original IBL

implementing three steps shown in Figure 1. Our[1] limits the irradiation of the light to virtual objects

technique requires HDR format for local and distant

only light sources in the distant image. We therefore

images, similar to Debevec’s technique [1].

should assume that there are no near light sources while

Figure 2 shows the processing flow of our technique.

using the original IBL. Conversely, when light sources

It first constructs the light source environment based on2

the illumination distribution obtained from the distantreflected to the chromium ball (specular surface ball) as

image, and near light sources obtained from the local

well as described in [1], as a distant image.

image, as Step 1 in Figure 2. It considers the result ofWe assume to use High Dynamic Range (HDR)

Step1 as initial condition of light energy, and thenformat for distant and local images. HDR format assigns

calculates the stable state of light energy using radiosity

more than 8bits for RGB values of each pixel, andmethod, as Step 2 in Figure 2. It finally generates

therefore it can finely represent the distribution ofsynthetic image referring the computed light energy, asillumination.

Step 3 in Figure 2.

Figure 5 shows an example of virtual objects we usedin our experiments. We assume that 3D geometry ofvirtual objects is given as triangular mesh.

Preparing local image, distantimage, and virtual objects

Step1
Constructing light sources
Figure 5: An example of virtual objects (mug)modeled as triangular mesh.

Step2

3.2 Step1: Constructing light sources

Calculating light energy

This section explains the step for constructing lightsources. Figure 6 shows an illustration which denotes
how we construct light source environment.

Step3
Image composition

Distant image mappedon a hemisphere

Local image
Virtual object

Figure 2: Processing flow of the presented technique.

BottomFigure 6: Constructing the light source environment
with two HDR images.

Figure 3: An example of the local image.
Figure 7: Mapping distant image onto a hemisphere.

In this technique, we suppose that virtual objects locate
around the center of a circular bottom in a spacesurrounded by a hemispherical wall from far away.
Figure 4: An example of the distant image.

Moreover, looking from an optional view point, weFigures 3 and 4 show local and distant images we usedsuppose to locate a local image in the back of virtualin our experiments. We assume that a local image shoots

objects. Also, the technique constructs distant lightnear light sources. We use the photograph of scenerysources by mapping a distant image onto the3

hemispherical wall modeled as triangular mesh, which

however, we think that we should explore a technique toenclose all virtual objects and the local image, as shown

automatically compute the preferable weight value.

in Figure 7. At this moment, the technique calculates

3.3 Step2: Calculating light energy

energy E1 of every triangles of the hemisphere, by

After assigning the initial energy E1 to the triangles of

averaging RGB values of pixels of the distant image

hemisphere and near light sources as described ininside each triangle. Here, E1 denotes the initial energy

Section 3.2, we calculate the stable values of the energyof light which is needed in Step 2. Also, it supposes E1 ofE2 of each triangle. Current our implementation applies

triangular elements of the floor as 0.

the radiosity method for calculating the energy. TheThe technique supposes that users design the geometry

technique calculates E2 twice: one is for the scene withof near light sources as triangular meshes as well as thethe virtual objects, and the other is for the scene without

hemispherical wall. We assume that the geometry

them.

consists of light and non-light parts (i.e. plastic stand).

Figures 9 and 10 show the results of distributions ofCurrently we manually design rough 3D geometry oflight energy calculated by radiosity method. Current ournear light sources using CG software, as shown in Figure

implementation provides an OpenGL-based viewer to8. Naturally, it is preferable to automatically reconstruct

show the radiosity results, which supports interactive

their 3D geometry from the local images. Here, there areviewing operation for manual calibration between themany techniques on reconstruction of 3D geometry fromlocal image and the radiosity results. If users have an

two or more images, in the field of stereo vision.

environment to shoot local images with marking multiple

Moreover, there are several techniques on manual

positions, we can automate the manual calibration.

reconstruction of 3D geometry, such as mapping
primitive geometry to objects in the photograph [4], inthe field of Image Based Rendering. Application of suchtechniques will be one of our future works.

Figure 9: Distribution of light energy
when a virtual object is in the scene.

Figure 8: Simple 3D geometry data of a near lightsource designed using CG software.

After designing the near light sources as triangularmeshes, the technique assigns the values proportional toRGB values of lights in the local image, to E1 of theFigure 10: the distribution of light intensity

triangular elements corresponding to the light parts. Also,

when a virtual object is not in the scene.

it assigns E1 as 0, to the other triangular elements

3.4 Step3: Image composition

corresponding to the non-light parts.

This section explains how to generate a synthetic image.

In order to appropriately balance the optical energy

At this step, new RGB values are computed based on

between the distant and near light sources, we have toRGB values of the local image, and energy E2 of eachassign the weights to the light sources, because their

triangle computed by the procedure described in Section

actual light energy do not always proportional to the3.3. This step assumes that virtual objects locate in front

RGB values in distant and local images. Currently weof real objects shot in the local image, and supposes that

explore a preferable weight value by try-and-error;

the virtual objects are not partially occluded by the real4

reflection etc., and to express the virtual object toobjects.
Similar to the original IBL described in [1], ourcompound with reality by applying well-known shading

technique determines whether local image or virtual

models such as BRDF (Bidirectional Reflectanceobjects is rendered, for each pixel of the synthesizedDistribution Function).

image. Here, current our implementation generates a4. Experiments

mask image, as shown in Figure 11, where black pixels

This section shows our experiments. We implemented

denote they are not occluded by the virtual objects.

and executed all steps on Windows XP and GNU gcc 3.4.

Simultaneously, it captures RGB values of each pixel of

We created HDR images by unifying the brightness

two images shown in Figures 9 and 10 by using thedistribution of three or more images taken underfunction glReadPixels().

different exposure to prepare distant image and local
image. Moreover, we designed 3D geometric models of
virtual objects and a near light source using 3D StudioMax, and prepared by saving this as triangular mesh data
in VRML format. After saving the image generated by
Step 3 in HDR format, we converted the image into
JPEG format by using commercial software and applying
tone mapping.

Figure 11: Mask image for determination ofeach pixel for image composition.

Our implementation calculates RGB values of pixels,
where are black in the mask image, from the RGB valuesof the local image. In this case, it calculates the RGB
values by multiplying the ration of energy before andafter locating the virtual objects by the RGB values ofthe local image, as the following equation:(RG(RG(R ' G '⎛ IR ⎞⎜ ⎟⎜ IG ⎟⎜ IB ⎟⎝ ⎠B ) = (R 'G'⎛ IR ⎞⎜ ⎟B ' )⎜ I G ⎟⎜ IB ⎟⎝ ⎠Figure 12: Example of an image withconsideration of the near light source.

B ) : RGB values of a synthetic image
B '): RGB values of an original image: The ratio of energy before and after
locating the virtual objects.

The calculation reflects the optical effects of virtualobjects, such as reflection of light energy from the virtualobjects to the floor, or shadows on the floor made by theFigure 13: Example of an image without

virtual objects.

consideration of the near light source.

Our implementation calculates RGB values of pixels,

Figure 12 shows example of an image generated bywhere are white in the mask image, by applying

our technique. We can observe the shadow of cup caused

well-known shading models; however, currently itby the illumination of stand light. Fig.12 shows thatconsiders only diffuse reflection. As future works, weconsideration of near light source is effective in IBL.

have to treat not only to diffuse reflection but to specular

We also generated a synthetic image without taking a5

near light source into consideration by using existingbrightness when users use distant image andmethod, as shown in Figure 13. While generating both

local image which were shot and edited underimages in Figures 12 and 13, fluorescent lights of adifferent conditions
The purpose of this report was to generate a realceiling of a room as distant light sources illuminated amug. Processing flows of image synthesis in Figures 12

syntheticand 13 were same, except E1 values of near light sourcesin this paper can be applied to PRT (Pre-computed

were 0 while generating the image shown in Figure 13.

Radiance Transfer) [5] for real-time realistic image

still

image; however, we think our conceptsynthesis.

Moreover, we actually placed the mug near the stand

References

light, and shot a photograph, as shown in Figure 14.

[1] P. Debevec, “Rendering Synthetic Objects into Real
Scenes: Bridging Traditional and Image-based Graphics
with Global Illumination and High Dynamic RangePhotography”, SIGGRAPH 1998, pp. 189-198, 1998.
[2] I. Sato, Y. Sato, K. Ikeuchi,

Seamless Integration ofComputer Generated Objects into a Real Scene Based ona Real Illumination Distribution , Journal of the InstituteofElectronics,

Information,andCommunication

Engineers, J81-D-II, No. 5, pp. 861-871, 1998.
[3] T. Okabe, Y. Sato, “Effects of Image Segmentation

Figure 14: Photograph which actually placed mug.

for Approximation Object Appearance under Near
Lighting”, MIRU 2005, pp. 80-87, 2005. 80-87, 2005.
[4] P. Debevec, C. Taylor, J. Malik, “Modeling andRendering Architecture from Photographs”, ACM
SIGGRAPH ’96, pp. 11-20, 1996.
Figure 15: Comparison of synthetic portions.

[5] P. Sloan, J. Kautz, J. Snyder, “Precomputed Radiance(left) From Figure 12. (middle) From Figure 14.

Transfer

2002, July, pp. 527-536, 2002

Figure 15 shows the zoomed portion of thevirtualobject

andtherealobject.

Considering of the near light source, our techniquerepresented the shadow near the virtual object very
similar to the real shadow observed in Figure 14. Thus,
we emphasize constructing light source environments ofthe scene more faithfully in order to generate a naturalsynthetic image.

5. Conclusion and future works
In this paper, we presented how to render virtualobjects into real scenes with near light sources using IBL.
The following points are our future works.
•Semi-automatic generation of the 3D geometryof near light sources to ease a burden of ausers•Real-TimeRenderinginDynamic,

Low-Frequency Lighting Environments”, SIGGRAPH(right) From Figure 13.

synthesizedforinput.

Re-examination about the consistency of the6