Proceedings of the Fifth IIEEJ International Workshop
on Image Electronics and Visual Computing 2017
Da Nang, Vietnam, February 28- March 3, 2017

HUMAN-BODY MODELING AND POSITION SPECIFICATIONFOR FORENSIC AUTOPSY DATA VISUALIZATION
Junki Mano†
Hideki Shojo††

Masahiro Toyoura†
Noboru Adachi†

University of Yamanashi

ABSTRACTThis paper proposes the methods for creating a humanbody model that reflects individual victim characteristics
and obtaining position information on the human-bodymodel for the purpose of forensic autopsy datavisualization. Although modern imaging technologies,
such as CT and MRI, make it possible to obtain finevolumetric human-body model, these technologies arestill not widely used in the real forensic autopsy sites dueto the cost. To enable a forensic specialist to easily create
3D models of victims, we propose a new method whichuses an affordable depth-sensing camera and requires few
user interactions. To visualize the autopsy data directly
on a 3D model, we also propose a method for specifying
a position on the 3D model without impeding the
progress of the autopsy procedure.
1. INTRODUCTION
Forensic autopsy reports play significant roles in court
decisions. It includes data on any injuries apparent on the
body subject to the autopsy, photographs of thecorresponding injury locations, and the findings of the
medical examiner who performed the autopsy. Forensic
autopsy reports contain substantial amounts of technicalterminology and unfamiliar statements. For jurors
without any background in legal constructs or autopsy
procedures, then, it can be difficult to understand thenature of the injuries. Computer graphics (CG)technology can be useful in making autopsy results moreintuitive, as they provide direct visual renderings of
injuries and other conditions on a 3D CG model of thevictim.
Figure 1 shows an example of the type of forensic
autopsy data visualization that this study addresses. The
visualization method involves associating injury data andphotographs from an actual forensic autopsy to injury
locations on a human-body model that reflects the unique
individual characteristics of the body subject to the
autopsy. Selecting a site on the human-body model brings
up the associated forensic autopsy data in a new window
and renders a direct visualization on the human-bodymodel via texture mapping.
This visualization approach requires two primarysteps: building a human-body model that reflects theindividual characteristics of the body subject to the
autopsy and obtaining position information on thehuman-body model, thus enabling the mapping of††

Xiaoyang Mao†
Issei Fujishiro††
Keio University

forensic autopsy data. Researchers have attempted tocreate human-body models using the “virtual autopsy”[1] approach, which involves performing a CT scan on
the body and analyzing the results on a computer. But
these technologies are still not widely used in the real
forensic autopsy sites due to the cost for purchasing andmaintaining 3D imaging instruments. To obtain forensic
autopsy data from the source in real time and associate
the data with a corresponding human-body model, one
would also need to be able to gather position information
during the autopsy without impeding the progress of the
autopsy procedure.
To address these needs, this study aims to propose amethod for using a depth-sensing camera and a pointing
device to create a human-body model that reflects
individual characteristics and allow users to obtain
position information on the human-body model during
the course of an autopsy.

Figure 1: An example of forensic autopsy data visualization
2. RELATED RESEARCH
Input system for forensic autopsy data
Boussejra et al. [2] proposed an input system for forensic
autopsy data. The system was built with a mark-up
language LMML (legal medicine mark-up language)
which is designed for describing forensic autopsy data. A
medical examiner can quicker and smoother input thedata than the conventional manual description. The
modeling and position specification technologies
proposed in this paper provides the key technology for
implementing the browser for viewing LMML data in 3D.

Proceedings of the Fifth IIEEJ International Workshop
on Image Electronics and Visual Computing 2017
Da Nang, Vietnam, February 28- March 3, 2017

Human-body modeling
Xi et al. [3] used non-numerical attributes like gender,
age, race, marital status, and occupation to predict body
shapes and build human-body models accordingly.
Although the research makes it possible for scientificcriminal investigators without any photographs of agiven suspect to get an estimation of the individual’s faceand body shape based solely on age, occupation, andother characteristics, the approach is not an effectivemeans of creating human-body models that reflectspecific individual characteristics. Anguelov et al. [4],meanwhile, proposed a method for creating human-bodymodels by using a training database of learned posturemodels. These approaches, however, are not effectivemeans of creating human-body models that reflectspecific individual geometric characteristics.
Allen et al. [5] used a standard model with an average
body shape and body scan data to create a human-bodymodel. As scan data is susceptible to the effects of noiseand data damage, Allen et al. began by preparing astandard model free of noise and incomplete surfaces andthen deformed the model in accordance with the scan data.
This made it possible to create a human-body model that
not only conformed to the shape of the scan data but also
eliminated the chances of any noise or incomplete
surfaces undermining the results. The Allen et al. [5]method, which enables the creation of human-bodymodels that feature individual characteristics and a fullbody mesh, thus provided a useful basis for our study.
Drawing on the Allen et al. [5] approach, we prepared astandard model and deformed it based on data readings
from a depth-sensing camera to create a human-bodymodel that reflects individual characteristics.
3D position specification
To obtain position information on the 3D human-bodymodel, thereby enable the mapping of autopsy data onto
the 3D model, we need a method allowing the user toeasily specify a position on the 3D model during theprocess of autopsy. As means for allowing interaction
between the real space and the there-dimensional space,
there are methods to use gestures and mobile devices [6][7]. These researches, as the means of interaction are
effective, but are not tailored for acquiring position
information on any 3D space.
3. PROPOSED METHOD
3.1. Overview
Figure 2 depicts the overview of the proposed method.
Before the forensic autopsy begins, we use Kinect RGBD sensor mounted on the ceiling of the autopsy room to
obtain a scan of the body subject to the autopsy and thenuse the scan data to create a human-body model thatreflects the individual characteristics of the body. Next,
the medical examiner proceeds with the forensic autopsy
and notes findings. When recording data or findings on agiven injury, the medical examiner specifies the positionof the injury by illuminating the site with a laser pointer.
The RGB-D sensor captures images of sites that theexaminer has specified, and our system thenautomatically converts that input into position
information on the human-body model.
Pointing device

RGB-D
sensorForensic
autopsy data

Figure 2: Overview of the proposed method
3.2. Human-body modeling
The proposed method generates a human-body model inthe following 2 steps:
1. Obtain scan data via an RGB-D sensor
2. Deform the standard model based on the scan dataobtained in step 1
Scan data from RGB-D sensors is susceptible toseveral problems, including noise that can occur during
the body scan, mesh collapse in areas that the scan fails
to measure correctly. Human-body models used in thevisualization of forensic autopsy data also need to retain
anatomic reference, which aid in the acquisition of
position information from the autopsy findings. Figure 3
shows the anatomic reference points.

Figure 3: Anatomic reference points
When recording position information for an injury in
his or her findings, the medical examiner uses theanatomic reference point(s) nearest the injury site to
denote the location. Reproducing these recorded findings
in a visualization would require a human-body modelwith the same anatomic feature points.
For these reasons, visualization cannot be generatedusing the scan data alone. Drawing on the Allen et al.
method [5], we thus deform a standard, predefined fullbody mesh model to fit the scan data. This makes it
possible to generate a noise-free, incomplete surface-free
human-body
model
that
reflects
individual
characteristics. By predefining anatomic reference points
on the standard model, our method also ensures that theresulting human-body models include anatomic
reference point information. As will be described later inthis section, we also predefine a skeleton on the standard
model for the purpose of posture registration

Proceedings of the Fifth IIEEJ International Workshop
on Image Electronics and Visual Computing 2017
Da Nang, Vietnam, February 28- March 3, 2017

The Allen et al. method [5] uses 3 error items, data
error, smoothness error, and marker error of Formula
1~3, respectively, to evaluate the degree of similarity
between the standard model and scan data.
𝑛

𝐸𝑑 = ∑ 𝑤𝑖 𝑑𝑖𝑠𝑡 2 (𝑇𝑖 𝑣𝑖 、𝐷)

(1)

𝑖=1

𝐸𝑠 =

∑

‖𝑇𝑖 − 𝑇𝑗 ‖{𝑖、𝑗|{𝑣𝑖 、𝑣𝑗 }∈𝑒𝑑𝑔𝑒(𝑚𝑎𝑡ℎ𝑐𝑎𝑙{𝑇})}𝑚
2
𝐸𝑚 = ∑‖𝑇𝑘𝑖 𝑣𝑘𝑖 − 𝑚𝑖 ‖
𝑖=1

2
𝐹

(2)

(3)

In Formula 1, each vertex vi in the standard model isinfluenced by a 4 × 4 affine transformation matrix Ti.
dist2() is the minimum distances between n vertices vi on
the standard model and the vertices on scan data D. We
then match the standard model with the scan data so thatthe total of the minimum distances is as small as possible.
wi is a weight for ensuring that the process does not link
non-matching vertices. If the minimum distances is large,
we decide that the correspondence between the points is
not reliable (for example there is missing data on scandata D) and we set the weight wi to zero.
For the smoothness error of Formula 2, we determine
the Frobenius norms of transformation matrices Ti and Tj
with vertices vi and vj, which are adjacent to each otheron the standard model, to calculate the amount of change.
We then implement controls to produce the deformation
with the minimum amount of change and thereby
maintain the mesh configuration of the standard model.
For the marker error function of Formula 3, the L2
norm for m marker points vki on the standard model andmarker point mi on the scan data are computed. We
perform matching to minimize the total of the distances.
The marker points in our study are the anatomic reference
points from Figure 3. The final error function E is thelinear combination of the three error items given above:
𝐸 = 𝛼𝐸𝑑 + 𝛽𝐸𝑠 + 𝛾𝐸𝑚(4)interfering with the initial deformation procedure. We
solve the problem by using the skeleton defined on thestandard model to align the postures between the standard
model and scan data first before starting the deformation.
Figure 4 illustrates the steps of the skeleton-based
initial deformation process.
1. Transform the control points of skeleton to thecorresponding positions in the scan data
2. Calculate the transformation matrix for converting
the bones of the skeleton on standard model to thebones of the skeleton obtained in step 1
3. Establish a transformation matrix applicable to thevertices of the standard model and then perform theinitial deformation
2.

1.

Figure 4: Overview of the skeleton-based initial
deformation process
To transform the control points of the skeleton on thestandard model to the corresponding points of the scandata, we use the transformation matrix obtained by
optimizing the marker error function 𝐸𝑚 . This isreasonable because the positions of markers (theanatomic reference points) are close to that of the control
points. Denoting a bone of standard model and its
correspondence in scan model as vi and vi’, respectively,
the translation matrix t, scaling value s, and rotation
matrix r for each bone can be computed via the following
formulas:𝑡𝑖 = 𝑣𝑖′ − 𝑣𝑖𝑠𝑖 =𝑟𝑥2 (1 − 𝑐𝑜𝑠𝜃) + 𝑐𝑜𝑠𝜃

By performing optimization processing to minimize
the result of Formula (4), we obtain transformation
matrix T for transforming the standard model. In Formula(4), α, β, and γ are weights for controlling the ratios of thethree error functions. We deform the standard model byapplying calculated transformation matrix T to thevertices of the standard model.
According to our experiments, using Allen’s method[5] with a low-resolution standard model cansuccessfully create a human-body model that closely
resembled the shape of the scan data but it may fail incase of a high-resolution standard model. We observedthat the reason for this problem is caused by thedifference of postures. When the standard model and thescan data have very different postures, the smoothness
error function 𝐸𝑠 falls into a local solution, thus3.

𝑟𝑖 = (𝑟𝑥 𝑟𝑦 (1 − 𝑐𝑜𝑠𝜃) + 𝑟𝑧 𝑠𝑖𝑛𝜃𝑟𝑥 𝑟𝑧 (1 − 𝑐𝑜𝑠𝜃) + 𝑟𝑦 𝑠𝑖𝑛𝜃𝑐𝑜𝑠𝜃 =(5)‖𝑣𝑖′ − 𝑣𝑗′ ‖(6)‖𝑣𝑖 − 𝑣𝑗 ‖𝑟𝑥 𝑟𝑦 (1 − 𝑐𝑜𝑠𝜃) + 𝑟𝑧 𝑠𝑖𝑛𝜃𝑟𝑦2 (1 − 𝑐𝑜𝑠𝜃) + 𝑐𝑜𝑠𝜃𝑟𝑦 𝑟𝑧 (1 − 𝑐𝑜𝑠𝜃) + 𝑟𝑥 𝑠𝑖𝑛𝜃𝑟𝑥 𝑟𝑧 (1 − 𝑐𝑜𝑠𝜃) + 𝑟𝑦 𝑠𝑖𝑛𝜃𝑟𝑦 𝑟𝑧 (1 − 𝑐𝑜𝑠𝜃) + 𝑟𝑥 𝑠𝑖𝑛𝜃)𝑟𝑧2 (1 − 𝑐𝑜𝑠𝜃) + 𝑐𝑜𝑠𝜃(𝑣𝑖 − 𝑣𝑗 ) ∙ (𝑣𝑖′ − 𝑣𝑗′ )‖𝑣𝑖 − 𝑣𝑗 ‖‖𝑣𝑖′ − 𝑣𝑗′ ‖

(7)
(8)

𝑠𝑖𝑛𝜃 = √1 − 𝑐𝑜𝑠 2 𝜃(9)𝑟𝑥(𝑣𝑖 − 𝑣𝑗 ) × (𝑣𝑖′ − 𝑣𝑗′ )(𝑟𝑦 ) =‖(𝑣𝑖 − 𝑣𝑗 ) × (𝑣𝑖′ − 𝑣𝑗′ )‖
𝑟𝑧

(10)

To deform the whole model at 3rd step, we need todetermine the transformation matrix to apply to eachvertex of the standard model. The transformation matrix
applied to each vertex should be computed based on themost relevant bone, for example based on thetransformation matrix of the closest bone.

Proceedings of the Fifth IIEEJ International Workshop
on Image Electronics and Visual Computing 2017
Da Nang, Vietnam, February 28- March 3, 2017

The geodesic distance is useful if want to find the
distance between two vertices on the mesh, but generallybones are defined in space rather than on mesh, so we
can’t use geodesic distance. Also, Euclidean distance canassociate a bone to a vertex on a different part of the body,such as associating an arm to the closest side of upper
body, so it is not possible to determine an appropriate
transformation matrix. Therefore, we first segment thestandard model using Huang et al. method [8] before
using Euclidean distances to ensure the correct
correspondence. Figure 5 shows the segmentation of thestandard model.

(a)Standard model(c)Initial result(b)Scan data(d)Final result(e)Final result(high resolution)

Figure 6: Modeling with the proposed method
Figure 5: Segmentation of standard model
We compute the Euclidean distances between thevertex and bone according to segmentation result, andassociate vertices with bones that are closest in distanceand determine the transformation matrix to apply to eachvertex. After that, we perform initial deformation using
skeletal subspace deformation to obtain a smooth result.
With the initial model available, it is further refined by
iterating the optimization process that minimize the error
function given in Formula (4). Since the resolution of thestandard model used is low, a low resolution model will
be constructed as the result. Finally, we increase theresolution of the resulting model by using Catmull and
Clark’s method [9]. This method for recursively
generating surfaces that approximate points lying-on
mesh of arbitrary topology, so it is possible to maintainthe topology of result model and increase the resolution.
Figure 6 shows a result of the proposed method. Figure6(a) and (b) are the standard model and scan data,
respectively. Figure 6(c) is the result of the skeletonbased initial deformation and Figure 6(d) is the final
result. As the Figure indicates, the initial deformation
process ensured that even a high-resolution standard
model with different posture would produce results thatapproximated the shape of the scan data. One can also seethat performing a detailed deformation using the initial
posture deformation results created a human-body modelwith an even closer resemblance to the shape of the scandata. Also, Figure 6(e) shows the result of increasing theresolution of Figure 6(d). As in Figure 6(d), it can be
confirmed the result is close to the shape of scan data.

3.3. Position Specification
Below is an overview of the process for identifying a
specified position on the human-body model.
1. Use an RGB-D sensor to detect specified positions
pointed by a laser pointer2. Convert the specified positions detected in step 1 tocoordinates on the human-body model
3. Obtain the position on the human-body model
To maximize operability in actual autopsy settings, weallow the medical examiner uses a laser pointer toilluminate and thereby specify the positions of sites
where he or she intends to record findings. By then using
an RGB-D sensor to detect the specified positions, ourmethod automatically calculates the corresponding
positions on the human-body model.
We detect a position that the medical examiner has
specified with the laser pointer using backgroundsubtraction technique. We obtain frame-by-frame images
via the RGB-D sensor, and then the position pointed canbe detected as the position with large difference from theprevious frame. Next we need to convert the position
coordinates in the RGB-D sensor coordinate system tothe position coordinates of the human-body model. To doso, we use a rigid ICP algorithm to register the predefinedskeleton for the standard model and the skeleton obtained
via RGB-D sensor and then obtain a transformation
matrix for converting coordinates from the RGB-D
sensor coordinate system to the human-body model
coordinate system.
Figure 7 shows the results of obtaining positions on
the human-body model. We can see that using a laser
pointer resulted in accurate specified positions and that

Proceedings of the Fifth IIEEJ International Workshop
on Image Electronics and Visual Computing 2017
Da Nang, Vietnam, February 28- March 3, 2017

the position information on the human-body model
conformed to the original specified positions.

(a) Pointed posion on amannequin(b) Computed position on
the 3D model

Figure 7: Results of obtaining position information on
the human-body model
4. CONCLUDING REMARKSThis study proposes a method for creating a human-bodymodel and obtaining position information on the humanbody model for visualizing forensic autopsy data. As
shown in Figure 6, although the resulting model presents
the shape characteristics of the scan data, the method
failed to produce the correct shape of hands. One possible
reason is that the smoothness error function 𝐸𝑠 may fall
into a local solution for the fingers. We will try to
introduce some structure aware method for solving thisproblem. Currently the algorithm has only been tested in
the laboratory by scanning a mannequin with a RGB-D
sensor. We plan to evaluate and improve the method byapplying it to real autopsy site.
5. ACKNOWLEGEMENT
This work was supported by JSPS Grants-in-Aid for
Scientific Research KAKENHI (JP26240015).
6. REFERENCES[1] Patric, L., Calle, W., Anders, P., Claes, L. and Anders Y.:
Full Body Virtual Autopsies Using a State-of-the-art VolumeRendering Pipeline, proceedings of TVCG, Vol. 12, No. 5, pp.
869-876(2006).
[2] Boussejra, M. O., Adachi, N., Shojo, H., Takahashi, R.,
Fujishiro, I.: LMML: Initial developments of an integrated
environment for forensic data visualization, EG/IEEE VGTC
Conference on Visualization (EuroVis Short Papers), pp.31-35(2016).
[3] Xi, P., Guo, H. and Shu, C.: Human Body Shape Prediction
and Analysis Using Predictive Clustering Tree, proceedings ofthe 2011 International Conference on 3D Imaging, Modeling,
Processing, Visualization and Transmission(3DIMPVT '11), pp.
196-203(2011).
[4] Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers,
J., Davis, J.: SCAPE: Shape Completion and Animation of
People. ACM SIGGRAPH, Volume. 24, Issue. 3, Pages. 408416(2005).
[5] Allen, B., Curless, B. and Popovic, Z.: The space of humanbody shapes: reconstruction and parametrization from range
scans, proceedings of ACM SIGGRAPH, Vol. 22, Issue. 3, pp.
587-594(2003).
[6] David, D., Jorg, S. and Sven, B.: Learning to interpret
pointing gestures with a time-of-flight camera, proceedings

HRI '11 of the 6th international conference on Human-robot
interaction, pp. 481-488(2011).
[7] Krzysztof, P., Anastasia, K., James R, W. and Edward L.:
Smartcasting: a discount 3D interaction technique for public
displays, proceedings of the 26th Australian Computer-Human
Interaction Conference on Designing Futures: the Future of
Design, pp. 119-128(2014).
[8] Huang, Q., Koltun, V. and Guibas, L.: Joint shapesegmentation with linear programming, proceedings of ACM
SIGGRAPH Asia, Vol. 30, Issue. 6, Article. No. 125(2011).
[9] Catmull, E., and Clark, J.: Recursively Generated B-Spline
Surfaces on Arbitrary Topological Meshes, Computer Aided
Design 10, 6, 350–355(1978).