The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx

Forensic CG Video Generation
with Augmented Reality Technology
Atsushi Sugiura(Non-member) Masahiro Toyoura(Non-member) Xiaoyang Mao(Member)Interdisciplinary Graduate School of Medicine and Engineering, University of Yamanashi{g12dhl02, mtoyoura, mao} ＠ yamanashi.ac.jp
Abstract
This paper presents a technology for supporting in-house forensic CG video generation by police andprosecution. For confidentiality, it is usually not allowed to generate criminal CG video through subcontracting,whereas existing CG modeling and video editing systems are too complicated to use for police or prosecution.
The proposed technique solves this problem with Augmented Reality (AR) technology. We propose to deploy
multiple cameras and AR markers on criminal site. While a user of the police or the prosecution plays a role ofthe criminal wearing a video see-through head mount display, video is automatically generated. CG objects, suchas the virtual victim, are displayed on the head mount display and rendered in the video. Our main contributionsin this research are concurrent displaying of first-person and third-person views for visualizing spatial
relationship between the user and its surroundings, and complementary use of marker information from multiple
cameras for solving the occlusion problem inherently caused by operating at the actual criminal site.

Keywords
Augmented Reality, Multi-Marker Complement, Multiple Views, ARToolKit.

1

The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx

1. Introduction

reducing network delay for AR applications are introduced in

The lay judge system has been introduced in many countries.

2.3.

The lay judges are chosen from citizens in general. They discuss

2.1 Crime Scene Modelingabout guilt or innocence and the appropriate punishment withprofessional judges. The system makes the judge to follow theCreating the 3D CG model of criminal site requires technicalsense of ordinary people. It also makes the period much shorter.

skills. It also costs much time. March et al. proposed a

Since the lay judges are amateur, the materials provided to themtechnique for converting forensic pathology descriptions into

are required to be easily understandable without expert

3D CG models [2]. Thali et al. generated individual models ofknowledge.

corpses using markers [3]. Captured images, MRI (Magnetic

We focus on the demonstration CG video as a part of forensic

Resonance Imaging) and CT (Computed Tomography) data arematerials, which enables the judges to understand the criminal

integrated to create the 3D CG model with the markers.

situation intuitively. Although it is already generally used in TV

Compared to the corpses, criminal site and the animation

programs, it has been rarely used in the court. This is because

depicting the process of a crime are much more difficult to

subcontracting is inappropriate for confidentiality, while police

model since they varies case by case and there are no standard

or prosecution is not expertise for generating movie. We

models are available. Se and Jasiobedzki [4] have proposed the

propose to support in-house CG video generation withinstant Scene Modeler (iSM), a 3D imaging system that

Augmented Reality (AR) technology. AR technology support toautomatically creates 3D models using an off-the-shelfdisplay CG objects in video captured real world. Supplementaryhand-held stereo camera. The user points the camera at a scene

information can be added in the video. An initial version of thisof interest and the system will create a photo-realistic 3D

paper was presented at NICOGRAPH international 2012 [1].

calibrated model automatically within minutes. Clair E et al. [5]We suppose multiple cameras and multiple markers areproposed to model the crime scenes using the sketch based

deployed at the criminal site. A user, such as the police or themodeling tool called SketchUp [6]. These tools are good atprosecution, plays the role of the criminal wearing a video

creating static CG models, but poor at creating dynamical

see-through Head Mounted Display(HMD). CG objects aremodels such as human action or moving weapons. Thoserendered in the video captured criminal site and displayed to the

dynamical models are vital for depicting a crime.

user with the HMD. In this way, modeling of human motion andWe solve the problem by using AR technology. In our system,

actual environment is not necessary, and all things the user

the CG model of victim, which can usually be obtained byshould do for generating CG video is playing the role of themodifying a standard model, is rendered into the captured image

criminal on site. AR markers may be occluded by the user andof crime site, and a video depicting the process of crime can beobjects in a real site. We solve the problem by complementing

automatically generated simply by having a user playing themarker information among multiple cameras. Displaying

role of criminal. CG objects can be arranged in the site

multiple views from the multiple cameras enables the user tointuitively by using AR markers.

grasp the spatial relationship between the user and his/her

2.2 AR Systems supporting multiple views

surroundings, even if the user is not well trained for using AR
system.

Supporting of two or more views in AR system has been

In the next section, we first introduce related works. Section 3

actively discussed. Sport is a good application of AR. The 3D

describes how the proposed system generates CG video. The

soccer observation system developed by Inamoto and Saito

results of subject studies are described in Section 4, and Section

generates free viewpoint video from images captured with a

5 concludes the paper.

large number of cameras [7]. A viewer can specify an arbitrary
viewpoint and the system generates the corresponding image by

2. Related works

interpolating multiple images from different cameras. Cameras

Several research groups have proposed to use CG models as

are calibrated in advance. The video generated by interpolating

forensic materials. Section 2.1 introduces the existing work on

multiple views gives virtual views, which is not suitable forcreating CG models based on the investigation data of

being used as forensic materials since the reliability is mostcriminals.

important for the judge. Our system generates third-person

We use multiple views to help the user understand the spatial

views from real cameras. Viewers can change the camera, andrelationship between himself/herself and surroundings. The

examine the situation from the viewpoints of different cameras.

system consists of multiple cameras and multiple PC connected

Integrating information of multiple views is another aim ofthrough a client/server network. Existing works on multiple

our system. Only from the first person view, the performer isviews AR are described in Section 2.2. Research works on

difficult to perceive the depth information. By displaying the2The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx
third-person views, we expect it would be easy for thecannot expect large equipment such as position sensors or aperformer to keep an intended posture. Thoung et al. [8]good lighting condition. We should detect AR markers in suchproposed an outdoor AR system for investigating the objects at

situation. The problem is solved by using multiple markers

a distance. They adopted zooming cameras in addition to thecaptured by multiple cameras in a complementary way. The

head mounted cameras. The system can provide the images ofmultiple cameras also support a user to act in the space. The

zooming cameras. Enomoto et al. [9] employed the cameras ofuser wears a HMD, and the images from multiple cameras areother points of views for diminished reality. By integrating thedisplayed on the monitor of HMD.

images from multiple viewpoints, an object behind a marker can

Section 3.1 describes the overview of our system, and Section

be transparently displayed in the first-person view. In our

3.2 describes the complement of information among multiple

proposed system, we expect the performer to integrate multiple

markers. Section 3.3 is about how to support the user byviews in real time. The system just provides multiple views,

displaying multiple views, and Section 3.4 introduces the

avoiding lowering the performance with the extra cost of

configuration of server/client network for transferring multiple

multiple view integration. For achieving such purpose, theviews.

position and size of the images on HMD should be carefully

3.1 Overview

designed and real time transfer of those images is also very
important.

We arrange multiple cameras and multiple AR markers on theactual crime site. We assume the lightings and the position

2.3 Reducing Network delay for AR applications

sensors are not available since the space is not a managed studio.

Research works combining AR and network technologies

The markers are often occluded by the user or other objects inhave also been conducted. The work by Okamoto et al. [10]real world. When multiple cameras are placed in the space, wetransmits and receives the information on the real world

can expect some of them can detect the marker. Then, marker

between remote sites and aimed at supporting all operations in

information

real-time. However, they did not address the problem of video

information from those cameras. In addition, the multiple

latency caused by network transfer, but focused on the latency

cameras provide multiple views supporting the user to grasp themainly related to AR applications. Lai and Duh [11] havespatial relationship among the virtual objects and real objects

reported that the latency affects dynamic quantitative perception

including the user himself/herself.

can

be

complemented

byintegrating

the

HMD with cameraof virtual objects. The delay problem has been well dealt in
Web camera

mobile AR applications [12]. Pasman and Jansen [13] proposedcoarse-to-fine representation of virtual objects for AR
applications. In their application, 3D CG models were rendered
in a mobile device. Mito et al. [14] transferred minified andcompressed images in JPEG format to reduce the network
latency in their application of immersive airship controlling.
However, they have not examined the effect of network latency

AR marker

for AR airship controlling.
Our system also performs the data communications between
personalcomputers

using

a

network,

considering(a) Before overlaid. The user wears HMD. The multiple

thecameras and markers are deployed in the space.
Virtual weapon

requirement of synchronized displaying of third-person view
images and first-person view images, the size of transferred
images should be small. We set the size of transferred image as
160×120 pixels, which is 1/16 of the original captured size. We
examine the effect of size reduction in Section 4.4.

3. Proposed method
Our method generates a video for visualizing a crime scene
with AR technology. The video consists of the images obtained

Virtual victim

by rendering the virtual objects, such as the 3D model of a(b) After overlaid. A video captured with multiple cameras. The

victim, weapon, over the real camera captured images ofvirtual victim and the virtual weapon are displayed on thecriminal site. The knowledge about CG modeling is not needed.

positions of the markers in the video.

When using the actual criminal site as the background, weFigure 1. Configuration of the proposed system.
3

The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx
The configuration of our proposed system is shown in Figure

When a marker is detectable, a geometric transformation

1. As shown in Figure 1(a), the user wears a HMD. The video of

matrix P can be calculated with the function of ARToolKit.

first-person view is displayed on the HMD. Multiple cameras

Then a virtual object can be displayed on the marker in thewhich are connected to the network are arranged in the space.

marker coordinate system.

They capture the video from the third-person viewpoints. The

Without loss of generality, let’s consider a system consists ofvideo is transferred to the user's HMD. A composite video withtwo cameras and two markers as shown in Figure 3. Denote theboth of the first-person view and the third-person views isgeometric transformation matrix of marker A and B detected

provided to the user.

from camera 1 are P1A and P1B, the geometric transformation

Multiple markers are placed in the space. The individual

matrix of marker A and B detected from camera 2 are P2A andmarkers correspond to individual virtual objects, with the one

P2B. Assume camera 1 can detect both marker A and B, but camera

for a virtual victim set on the floor, and the one for a virtual

2 can detect marker B only. Because marker A is undetectableweapon grabbed in the user's hand. The virtual victim and thefrom camera 2, geometric transformation matrix P2A cannot be

virtual weapon are displayed on the positions of the markers incalculated. Assume the camera coordinate system of camera 1

the video. It means that the video captured in the actual crime

viewed from camera 2 is Φ2→1, then the transformation matrixsite is augmented with the virtual objects by¥ AR technology.

of marker B viewed from camera 2 via camera 1 becomesΦ2→1･P1B. That is, we can obtain the following equation:

3.2 Complement of multiple marker information

P2B = Φ2→1 ・P1B .

ARToolKit [15] is a library for creating AR environment
easily. It detects a marker with a camera, estimates the position(1)

Similarly, the transformation matrix for maker A viewed

and posture of the marker, and displays a virtual object on thefrom camera 2 via camera 1 isposition indicated by the marker. However, when a part of themarker is absent from the field of view of the camera or when

P2A = Φ2→1 ・P1A .

the mark is in an inappropriate angle relative to the camera, theCombining equation (1) and (2) we obtain the following

marker becomes undetectable. We solve the problem by using(2)

equation:

the information from multiple cameras complementary.

P2B = Φ2→1 ・P1B .

Figure 2 shows the flow of the complement of marker
information.

(3)

Since P1A, P1B, and P2B are available, P2A can be computedwith equation (3) and the marker coordinate system of marker A
viewed from camera 2 can be calculated from P2A.

2→1

Camera 1

P1B
P1A

Camera 2

P2A(undetectable)
1

1

1

P2B
1

Figure 3. Marker information complement. The projection
metrics by the two cameras and two markers can be calculatedby the complement of marker information.
The principle behind the above calculation is that we can firstobtain the spatial relationship between the two cameras throughthe marker which is detectable by both cameras (Equation (1))
Figure 2. Complement of undetectable marker information.

and then compute the coordinated system of the occluded

When camera 1 fails to detect a mark, a geometricmarker from its coordinate viewed from the other camera.

transformation matrix is calculated using the information from

Therefore, to detect an occluded marker, at least one morecamera 2.

marker should not be occluded from both cameras. Moreover,4

The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx
there is no way to complement a marker which is occluded fromareall cameras.

QCam-200SX) connected. They capture the third-person views.

In the proposed system, a network of client/server model isclientswith

webcameras(Logicool

C920

and

The video were 640×480 in resolution and 30fps in frame rate.

constructed for communicating among cameras. Marker

Two markers are placed in the space, one of them represents

information is detected by the client and sent to a server. A

the position of a virtual victim, and the other the position of aserver receives data from a client, when there is a marker which

virtual weapon. Figure 5(a) shows the marker for the virtual

was undetectable, the server complements marker information

victim, and Figure 5(b) the marker for the virtual weapon. The

by using the data from other clients.

size of both markers is 80mm×80mm. If the marker for thevirtual victim is placed on the floor, it rarely came into the sight

3.3 Displaying multiple views to the userof the subject. Therefore, the marker was placed on a stand as

When creating a synthetic image, the spatial relationshipshown in Figure 5(a). We designed the weapon as shown in

between a real object and a virtual object becomes important. It

Figure 5(b). The edge of the weapon was rendered as a virtual

may be difficult for a performer to precisely grasp the spatial

object.

relationship only with the image of a first person viewpoint. We
improve the situation by providing the performer the image of a third
person viewpoint. Since the performer can see the image ofvarious viewpoints, it becomes easier for him/her to understandthe spatial relationship among virtual objects and actual
criminal site. The multiple views are also useful for helping the

Client 1

user to confirm whether the numbers and positions of cameras
Client 2

and markers are appropriate during the setup of the environment,
which is usually a difficult task especially when the actual crimesite has a complicated structure.

Server

3.4 Network Data transfer between cameras
A network is used in order to transmit images of two or more

Figure 4. Environment for experiments. A server PC with HMD

viewpoint between personal computers. A server/client modeland two clients PCs with web camera are deployed in a room.

using TCP socket connection is used. The image of the third

The marker information is acquired from the cameras.

person view acquired with the camera of a client computer issent to a server via network. If the size of the images beingtransferred is large, the system can become slow depending on the
traffic of network. However, real time processing is very
important in case of an AR system. We solve the problem byusing small images for the third-person view. The downsizing is
executed before the images are transferred.

(a) Victim marker(b) Weapon marker

Figure 5. Markers used in experiments. The size of both markers is

4. Experiments

80mm×80mm. The height of the victim marker is 1.1m. A handle is

We examined whether the third person view can help subjects

attached to the weapon marker.

grasping the spatial relationship between the subjects and virtual

4.2 Complement of marker information

objects. Complement of marker information between multiple
cameras and multiple markers is also examined through anotherIn this experiment, we examined whether the complement ofsubject study.

marker information based on the server/client communication
was successful.

4.1 Environment

An operation scene is shown in Figure 6. The user wore a

Experimental environment is shown in Figure 4. Three PCs

HMD and grabbed the weapon marker in his hand. The cameras

are deployed in a room. A HMD with 2 cameras (Wrap920AR,on the HMD captured the video of first-person view. The video

Vuzix Corporation) is connected to a server PC. The resolutionof the third-person view from other cameras were transferred toof cameras is 640×480 and frame rate is 30fps. The resolution

the server PC.

of display monitors is 800×600. The HMD provides the video

The victim marker was arranged in the detectable space fromof the first-person view to the subject. The remaining two PCs

all cameras. In several frames, the weapon marker was out of
5

The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx
sight in first-person view. The result of marker complement isshown in Figure 7.

(a) First-person view.
Figure 6. Operation scene. The user wore a HMD and grabbed
the weapon marker in his hand. Three cameras capture theaction.

(b) Third-person view.
Figure 8. Complement of marker information based on thevictim marker. The marker information is calculated using thevictim marker of both first-person view and third-person view.
(a) Result without the complement of marker information. Since
the weapon marker went out of sight of HMD cameras, theFigure 7(a) shows the result without marker complement.

virtual weapon could not be displayed.

Since the weapon marker went out of sight of HMD cameras,
the virtual weapon could not be displayed. Figure 7(b) shows
the result with marker complement. The virtual weapon could
be displayed even if the marker went out of the view. The
marker information from the client computer cameras wastransferred to the server PC. The information of undetected
weapon marker was complemented by integrating marker
information detected in other cameras.
Figure 8(a) represents the first-person view, and Figure 8(b)the third-person view. Based on the victim marker, which could
be detected in both two cameras, the weapon marker isdisplayed in its position although the marker could not be
detected in the camera of the first-person view. The information(b) Result with the complement of marker information. The

about the spatial relationship between the victim marker and theinformation of undetected weapon marker was complemented

weapon marker is transferred through the network. As the result,by integrating marker information detected in other cameras.

the virtual weapon could be displayed in the first-person view.

The virtual weapon could be displayed even if the marker went

We examined the scene that the subject stabbed the virtualout of the view of HMD cameras.

victim with the virtual weapon. Whether the virtual weapon was

Figure 7. Marker information complement.

detected or not is shown in Figure 9. The marker could not be
6

The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx
detected in 765 frames without complement of marker

To furthermore examine the relationship between theinformation (Figure 9(a)), but could be detected in all framesdifficulty of tasks and the performance of the user, we set

with the complement of marker information (Figure 9(b)). We

another target angles of stabbing. We had the subjects stab theconclude that the complement was effective for displaying thevirtual cube in each of the following 4 angles: 30 degree right,

marker for CG video generation in our proposed system.

45 degree right, 30 degree up and 45 degree up. The error andtime were measured at the time that the virtual weapon touched

Objectthe cube. The average error and time of the tasks are

No complementsummarized in Table 1. According to Table 1, third-person

Onviews could help the subjects to set the virtual weapon at morecorrect angle in shorter time.

Off0

100 200 300 400 500 600 700 800 900[frame]The size of the cube was also changed between 100mm and
250 mm on a side to confirm the relationship between the size(a) The result without the complement of marker information.
ObjectComplement

and the performance of subjects. The results are shown in Table
2. In both cases, the subjects could set the virtual weapon at

No complement

more correct angle in shorter time. The larger targets tend to beOn

stabbed in shorter time, but the accuracy of the position had no
big difference relative to the size of the target.

Off0

100 200 300 400 500 600 700 800 900[frame](b) The result with the complement of marker information.
Figure 9. The result of marker detection with/withoutcomplement of marker information.

4.3 User support by displaying multiple views
In this experiment, we examined whether displaying multiple
views is effective for helping user grasping spatial relationshipbetween the virtual and real objects.
2 male subjects and 1 female subject in their 30's joined theexperiment. Figure 10 shows the image shown on HMD. The
third-person views are not displayed in Figure 10(a), butdisplayed in Figure 10(b).

(a) Without third-person view. Grasping spatial relationship is

From the comments of all subjects, displaying third-person

difficult with first-person view only.

views helped to easily grasping the spatial relationship between
the virtual objects and real objects including the subjects
themselves. As the task we had the subjects touch a specified
point with the tip of the virtual weapon. A 250mm×250mm×250
mm cube was displayed on the victim marker. The first task was
to put the tip of the virtual weapon on the surface of the cube.
The second task was to put the tip at 50mm distance away fromthe surface of the cube. We examined the accuracy and time tocomplete the tasks with and without third-person views
displayed. When the virtual objects are 300mm away from eachother, we start to measure the time for the tasks. We stop timing
when the subjects stopped the motion. The scene of the tasks isshown in Figure 12.

(b) With third-person views. The third-person views are added

Results are shown in Figure 13 and Figure 14. The error ofto the upper right of the image displayed to the HMD. Grasping

distance between virtual objects became smaller by displaying

spatial relationship become easier by referring to the third

the third-person views. The time for the tasks also became-person views.

shorter. Visualization of the third-person views was confirmed

Figure 10. Images displayed to the HMD.

to be effective for the tasks.
7

The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx
A problem found in experiments was that a planer weapon

300mm

marker as shown in Figure 15(a) may not be detected depending
on its orientation. As shown in Figure 15(b), we solve theproblem by arranging five markers on a cube. As the result, it
can be robustly detected from any direction and the subject(a) Before Task

could play a role of the criminal without paying special
0mm

attention to the marker orientation.
We also considered the position of the third-person views in
the display monitor. Lower right area often overlapped with theMoveright hand of the users. Therefore, for the right-handed users,
we use the upper right area of the monitor.
(b) Task 1

The relative distance and angle between the weapon andvictim are also displayed on the monitor as shown in Figure 16.

50mm

In real application, a user can use this function for checking theconsistency of current positions to those of an autopsy report.
Move

The distance and angle are calculated from the positions of theircorresponding markers. A user can select whether to display the(c) Task 2

information on the monitor or not.

Figure 11. Tasks for evaluating the effectiveness of displaying
third-person views.

Error of distance (mm)Time (s)

80

40

60

30

40

20

20

10

0
Without
With
third-person views0
Without(a) Error of distance(a) With third-person view.

With

third-person views(b) Time

Figure 13. Result of task1.
Time (s)Error of distance (mm)
80

40

60

30

40

20

20

10

0

0
Without
With
third-person views(b) An example of third-person view.
Figure 12. Experiment scene. We evaluated whether the(a) Error of distancesubjects could grasp the spatial relationship. The virtual weapon

Figure 14. Result of task2.

was brought close to a cube. The time and distance to the target
were measured.
8

Without
With
third-person views(b) Time

The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx
Table 1. Result for different stabbing angles.
Error of Angle (°)

We first measured the time for transferring the images of
640×480, 320×240 and 160×120, respectively. Table 3 shows

Time (s)Third-person views

Without

With

WithoutWith

the result. The time was measured as the average of 10 images.

30° right

4.82

2.79

16.24

15.27

They include compression time in a client computer, which is

45° right

6.48

1.97

18.04

15.4

almost linear to the number of pixels, and was ignorable.

30° up

4.78

2.23

16.82

15.38

45° up

6.14

2.08

17.9

16.12

Table 3. Delay time when images of 640×480, 320×240 and
160×120 are transferred through our network.
Image size

delay time (s)640×480

0.64

With

320×2400.28

160×120

0.08

Table 2. Result by varying target size.
Error of Dis (mm)Third-person views

Without

With

Time (s)Without

100mm

31.8

9.67

33.91

24.9

250mm

33.96

12.37

19.94

14.87
Next, we had the subjects perform the stabbing task of theexperiment described in Section 4.3. The accuracy and time for
640×480 and 160×120 are shown in Figure 17. The larger thetransferred images are the longer the task completion time is.
This is because the network latency makes it difficult for the
subjects to determine the position of the virtual weapon relative
to the virtual objects. Figure 17 shows that the subjects could(a) A single planar marker

set the virtual weapon at more correct position in case 160×120(b) Markers on a cube

Figure 15. Marker for the virtual weapon. A single planar

images were transferred and displayed. This demonstrates thatmarker often get missed from third-person view cameras. Bytransferring downsized images is effective and sufficient for

arranging 5 markers on a cube, the subject could play a role ofsupporting accurate operations.

the criminal without special attention to the marker direction.

Error of distance (mm)Time (s)
50

40

40

30

30
20
20
10

10

0

0
640×480 160×120(a) Error of distance

640×480

160×120(b) Time

Figure 17. Comparison of accuracy and completion time fordifferent images sizes.
Figure 16. Distance and angle information displayed. The user

4.5 Remarks by police

can check the consistency of current positions to those given by

We discussed with local police about the potential of using

an autopsy report.

our system in real situation. To the best of their knowledge,
currently no such techniques are available and Judges are

4.4 Network delay and image size
We propose to transfer third-person view images in smaller

preceded mainly with descriptions and photographs. They thinksize for avoiding network delay. Since the images are

an AR system is particularly useful for stabbing cases since it istransferred through general LAN, it takes longer time when

impossible to reenact the injury inside the victim's body. On theimages of larger size are transferred. The network delay may

other hand, they also pointed that functions for guaranteeing the

make it difficult for the user to positioning virtual weaponsconsistency between the generated video and the fact should be

accurately.

provided. Taking the stabbing cases as the example, the angle,
speed and depth of stabbing are very important information for
9The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx
validating the evidence, and should be visualized in consistency[4] S. Se, P. Jasiobedzki, “Instant Scene Modeler for Crime

with autopsy report or other evidences. Improving the system

Scene Reconstruction,” VCPR, vol.3, pp.123-131, 2005.

based on those remarks is one of our major future research[5] E. S. Clair, A. Maloney and A. Schade, "An Introduction to

directions.

Building 3D Crime Scene Models Using SketchUp," Journal on
the Association for Crime Scene Reconstruction, vol.18, no.3,

5. Conclusions

pp.29-47, 2012.

In this paper, in-house 3D CG video generation supporting[6] Trimble, SketchUp, http://www.sketchup.com/.

system using AR technology is proposed. The system focuses[7] N. Inamoto and H. Saito, "Fly-Through Observation Systemon the reenactment of the crimes causing injury or death, which

for 3D Soccer Movie Based on Viewpoint Interpolation,"

are impossible to reenact actually. PCs connected with a

Journal of The Institute of Image Information and Television

server/client network provide both of the first-person view andEngineers, vol.58, no.4, pp.529-539, 2004.

the third-person views on the HMD. It supports users to grasp[8] T. N. Hoang and B. H. Thomas , "An Investigation of

the spatial relationship between themselves and surroundings.

Camera Position, Visualization, and the Effects of Sensor Errors

The marker information detected from multiple markers isand Head Movement," ICAT, 2011.

integrated in the server PC. When a marker is occluded from a[9] A. Enomoto and H. Saito, "Diminished Reality using

camera, its information can be obtained from another camera.

Multiple Handheld Camera," ACCV, pp.130-135, 2007.

As a limitation of our current system, the victim does not[10] Y. Okamoto, I. Kitahara, Y. Ohta, "3D Human Body

react to the user. Actual victim may escape or be stunned, go

Display by Using Depth Information for Remote Shared Mixed

down when the weapon is approaching. The victim of ourReality," IEICE Technical Report, vol.94, pp.830-838, 2011.

current system just stands up on the marker. We can replace the[11] W.Y. Lai, H.B.L. Duh, "Effects of frame rate forvictim with captured data in advance. Timing of reaction can bevisualization of dynamic quantitative information in adefined by the approaching distance and angle of the user.

head-mounted display," SMC, vol.7, pp.6485-6490, 2004.

Modeling the reaction of the victim would improve the reality[12] G. Papagiannakis, G. Singh, N.M. Thalmann, "A survey ofof the generated video.

mobile and wireless technologies for augmented reality

As a future work, the interface for augmenting additional

systems," Computer Animation and Virtual Worlds, vol.19,

information to the video is desired. It should support the

no.1, pp.3-22, 2008.

visualization of information in the video. Recently, CT and MRI[13] W. Pasman, F. W. Jansen, "Distributed Low-Latency

have been introduced to judicial autopsy. Such data will

Rendering for Mobile AR," IMAR, 2001.

facilitate the generation of precise 3D models. We plan to[14] H. Mito, K. Yamazawa and N. Yokoya, "Support Systemprovide the technologies for calibrating CG models to real

for Controlling an Unmanned Airship Using Visual Information

objects.

Based on Mixed Reality," IEICE TechnicalReport, vol.108,

As another potential application, the lay judges can not only

no.432, pp.163-168, 2009.

watch the video, but experience the criminal site themselves by[15] H. Kato and M. Billinghurst, "Marker tracking nad hmd

using the AR system themselves, so as to understand thecalibration for a video-based augmented reality conferencing

situation of the crime better.

system," International Workshop on Augmented Reality,
pp.85-94, 1999.

References[1] A. Sugiura, M. Toyoura, and X. Mao, "Forensic CG Video
Generation with Augmented Reality Technology," NICOGRAPH International, pp.94-101, 2012.
[2] J. March, D. Schofield, M. Evison, and N. Woodford,"Three-Dimensional

Computer

Visualizationof

Forensic

Pathology Data," The American Journal of Forensic Medicine
and Pathology, vol.25, pp.60-70, 2004.
[3] M. Thali, M. Braun, U. Buck, and E. Aghayev, "ScientificDocumentation, Reconstruction and Animation in Forensic:

Atsushi SUGIURA received the B.S. and M.S. degrees in

Individual and Real 3D Data Based Geo-Metric Approachengineering from University of Yamanashi, Japan in 2002 and 2012,

Including Optical Body/Object Surface and Radiologicalrespectively. He is a Ph.D. student in Interdisciplinary Graduate School

CT/MRI Scanning," Journal of Forensic Sciences, vol.50, no.2,of Medicine and Engineering, University of Yamanashi, Japan. His

2005.

research interests include augmented reality and forensic visualization.
10

The Journal of the Society for Art and Science Vol. x, No. x, pp. xxxx-xxxx

Masahiro TOYOURA received the B.S. degree in Engineering,
M.S. and Ph.D. degrees in Informatics from Kyoto University in 2003,
2005 and 2008, respectively. He is currently an assistant professor at
Interdisciplinary Graduate School of Medical and Engineering,
University of Yamanashi, Japan. His research interests are augmented
reality and computer/human vision. He is a member of IEEE
Computer Society.

Xiaoyang MAO received her B.S. in Computer Science from
Fudan University China, M.S. and Ph.D. in Computer Science from
University of Tokyo. She is currently a professor at Interdisciplinary
Graduate School of Medical and Engineering, University of
Yamanashi, Japan. Her research interests include texture synthesis,
non-photo-realistic rendering and their application to scientificvisualization. She is a member of ACM SIGRRAPH and IEEE
Computer Society.

11