A Final Report, MBTC 1092

Submitted to

Mack-Blackwell Transportation Center

Kelvin C.P. Wang and David XY Li

Department of Civil Engineering

University of Arkansas, Fayetteville

July 26, 1999

# Abstract

A Multimedia based Highway Information System (MMHIS) is described in
the paper to improve the existing photologging system for various
operation and management needs. The full digital, computer based MMHIS
uses technologies of video, multimedia data synchronization,
three-dimensional visualization, high-speed networking, and video
server. This paper presents the development for the three-dimensional
(3-D) terrain visualization user-interface of MMHIS, based on the
Application Programming Interface (API) of OpenGL. Digital Elevation
Models (DEM) from the US Geological Survey (USGS) were used to construct
the 3-D terrain surface of the entire state. A unique picking algorithm
was devised to conduct visual queries on the terrain surface of the
state. The 3-D user-interface allows the user to zoom, rotate, and pan
around the statewide terrain surface for multimedia data query. Based on
this visualization technology, a number of transportation related tasks
can be conducted visually and interactively at real-time for
transportation planning, engineering design, public hearing, and highway
management.

# Acknowledgments

Support for this project was provided by a Department of Transportation
grant to the Mack-Blackwell Transportation Center and by the Arkansas
Highway and Transportation Department.

# report summary

Two types of engineering information are used in highway departments for
various studies: (1) video images collected through photo logging, and
(2) site data. The video images provide visual information for pavement
management, highway signing and marking improvement, and accident
analysis. The site data may contain some or all of the information on
construction and rehabilitation history, pavement layer information,
pavement width and type, Average Daily Traffic (ADT), accident history,
signing and marking inventory, and road geometric data. The limitations
of many existing systems are apparent in the areas of accessibility,
search capability of the video library, and synchronization of video
data with engineering site data. Some of the existing systems also lack
the capability of simultaneous multiple-user-access to high-resolution
video and site data. Due to the analog nature of the video signals, it
is difficult to integrate the visual information with computer
databases. In summary, it is a time consuming process to look up and
reassemble the video and site data in different formats and from various
sources.

To improve the usage of the photo logging system and provide better
tools for highway management systems, a Multimedia Based Highway
Information System (MMHIS) was developed. This multimedia service
employs new technologies in digital video, high-speed networking and
relational databases, so that users can look at photo logging video and
corresponding data sets simultaneously without leaving their offices. A
data synchronization algorithm was developed to dynamically display
motion digital video with traditional engineering data sets. The data
sets can contain any data records, such as as-built data, pavement
condition and performance, traffic safety, geometric features and other
infrastructure data. MMHIS can be used by not only traditional users of
photo logging systems, but also other engineering and planning
professionals.

An important element in MMHIS is the user interface. As all inventory
data from photo logging are location-specific and have spatial
characteristics, it was determined that an efficient and visual
interface is needed for users to conduct query runs and visualize
terrain features for planning and management purposes. In addition, it
will be beneficial to have statewide terrain surface at high resolution
for examination in public hearings and management meetings. Furthermore,
recent technological advances in both visualization hardware and
three-dimensional (3-D) software allow the development of these
capabilities. For instance, it is possible to obtain 1-meter resolution
satellite imagery for terrain models.

Terrain visualization is frequently used in military applications and
high-end commercial areas. For decades, stereo photogrammetry has been
widely used in highway departments for roadway design and improvement.
However, all these applications require high capital investment. For
stereo photogrammetry, only a limited number of stations are needed in a
department and they are not used for any management purposes. However,
the primary goal of using a terrain based user interface for MMHIS is to
develop a low-cost solution, so that users can efficiently use the
interface in a PC environment to query infrastructure data routinely
from their offices.

This 3-D user interface should allow the user of MMHIS to do the
following:

1.  Users can interact with a 3-Dimensional, color digital map for the
    entire state on a personal computer.

2.  Users can select data queries from this 3-D map based on visually
    available information on highway and road network.

3.  The visualization interface allows the user to rotate, zoom in and
    out, and pan around the terrain surface of the state.

Currently, the data sources for MMHIS are from an existing highway data
vehicle and USGS. It requires less than one full-time employee to
maintain the operation of MMHIS. However, a computer network and
workstations are needed to support a multi-user environment, as digital
video files have to reside in a central location due to their large
sizes. Video decompression of MPEG-2 support is also necessary of the
workstations. This feature is becoming a standard for many new
computers. 3-D visualization hardware is recommended for the
workstations, but not required. 3-D rendering can be conducted by the
CPU, but with slower speed than a computer can do with dedicated 3-D
hardware.

The report is organized into three portions: (1) introduction to the
capabilities of MMHIS and system design, (2) data sources and data
preparation for the 3-D model in MMHIS, and (3) 3-D visualization
methods and their implementations. Wang et al (1998) describes other
algorithms and developments in MMHIS, including the synchronization of
video and location data and database structure, which are not discussed
in this report.

MMHIS provides multimedia data viewing capabilities to highway
engineers. It allows a highway agency to examine road and roadside
structures without taking certain field trips. In addition, MMHIS in a
computer network environment can be used in communicating design and
improvement ideas among engineers and managers, and to the general
public. Future work includes the use of higher resolution data from
satellite imagery vendors and USGS.

# 

# table of contents

**Introduction** 1

**Capabilities of MMHIS and System Design of the 3-D Visualization** 3

**Data Sources and Data Preparation for the 3-D Visualization** 5

MMHIS 3-D Map Internal Data Formats for DEM 5

**3-D Visualization Methods and Their Implementations** 7

> Terms in 3-D Transformation 8
>
> View and Map Transformations in MMHIS 9
>
> Rendering of 2-D and 3-D Terrain Maps 13
>
> Picking Operation 14

**Conclusions** 16

**References** 17

# List of Figures

Figure 1. 3-D Map of Interface for MMHIS 18

Figure 2. User Selectable Turning Movements 19

Figure 3. System Level Design for MMHIS 20

Figure 4. Illustration of Projection of the Raster Image onto the
Terrain Surface 21

Figure 5. Modeling Transformations in MMHIS 22

Figure 6. Customized Viewpoint Position and Projector 23

Figure 7. Map Rendering in 2-D 24

Figure 8. Partitioning with a Triangular Mesh 25

Figure 9. Picking Operations 26

Figure 10. Picking Algorithm 27

# Introduction

Two types of engineering information are used in highway departments for
various studies: (1) video images collected through photo logging, and
(2) site data. The video images provide visual information for pavement
management, highway signing and marking improvement, and accident
analysis. The site data may contain some or all of the information on
construction and rehabilitation history, pavement layer information,
pavement width and type, Average Daily Traffic (ADT), accident history,
signing and marking inventory, and road geometric data. The limitations
of many existing systems are exhibited in the areas of accessibility,
search capability of the video library, and synchronization of video
data with engineering site data. Some of the existing systems also lack
the capability of simultaneous multiple-user-access to high-resolution
video and site data. Due to the analog nature of the video signals, it
is difficult to integrate the visual information with computer
databases. In summary, it is a time consuming process to look up and
reassemble the video and site data in different formats and from various
sources.

To improve the usage of the photo logging system and provide better
tools for highway management systems, a Multimedia Based Highway
Information System was developed. This multimedia service employs new
technologies in digital video, high-speed networking and relational
databases, so that users can look at photo logging video and
corresponding data sets simultaneously without leaving their offices. A
data synchronization algorithm was developed to dynamically display
motion digital video with traditional engineering data sets. The data
sets can contain any data records, such as as-built data, pavement
condition and performance, traffic safety, geometric features and other
infrastructure data.

Connecticut Department of Transportation (ConnDOT) has been using an
analog based laser disk based photo logging system for many years
(Hanley and Larson, 1992). The original system is still actively used at
ConnDOT, which recently started incorporating digital images into the
photo logging system. Several other state highway departments are also
studying the application of digital images and video for either just
photo logging or an integrated information system. Based on the
experiences from several existing applications, MMHIS was developed from
ground-up into an integrated system that incorporates digital motion
video, a customized database engine, tight data synchronization, and
geo-referencing capabilities. MMHIS can be used by not only traditional
users of photo logging systems, but also other engineering and planning
professionals.

An important element in MMHIS is the user interface. As all inventory
data from photo logging are location-specific and have spatial
characteristics, it was determined that an efficient and visual
interface is needed for users to conduct query runs and visualize
terrain features for planning and management purposes. In addition, it
will be beneficial to have statewide terrain surface at high resolution
for examination in public hearings and management meetings. Furthermore,
recent technological advances in both visualization hardware and
three-dimensional (3-D) software allow the development of these
capabilities. For instance, it is possible to obtain 1-meter resolution
satellite imagery for terran models.

Terrain visualization is frequently used in military applications and
high-end commercial areas. For decades, stereo photogrammetry has been
widely used in highway departments for roadway design and improvement.
However, all these applications require high capital investment. For
stereo photogrammetry, only limited number of stations are needed in a
department and they are not used for any management purposes. However,
the primary goal of using a terrain based user interface for MMHIS is to
develop a low-cost solution, so that users can efficiently use the
interface in a PC environment to query infrastructure data on routine
basis from their offices.

This 3-D user interface should allow the user of MMHIS to do the
following:

4.  Users can interact with a 3-Dimensional, color digital map for the
    entire state on a personal computer.

5.  Users can select data queries from this 3-D map based on visually
    available information on highway and road network.

6.  The visualization interface allows the user to rotate, zoom in and
    out, and pan around the terrain surface of the state.

The paper is organized into three portions: (1) introduction to the
capabilities of MMHIS and system design, (2) data sources and data
preparation for the 3-D model in MMHIS, and (3) 3-D visualization
methods and their implementations. Wang et al (1998) describes other
algorithms and developments in MMHIS, including the synchronization of
video and location data and database structure, which are not discussed
in this paper.

# Capabilities of MMHIS and System Design of the 3-D Visualization

Figure 1 shows the main user screen of MMHIS. The highway video is
presented along with the corresponding site data table left to the
video. The site data table includes location data on the road being
viewed. The two windows of graphs show roughness and rutting data. The
capability of dynamic graphing is built into the two graphing windows.
This capability allows the user to view the values of various
attributes, such as roughness and rutting at the viewed location against
values for the entire road section. Two vertical bars on the graphs
indicate the location of the road being viewed relative to the whole
road section.

The map shown at the left-hand side of Figure 1 is geo-referenced 3-D
terrain map that can be used as a query tool. The actual terrain of the
State of Arkansas is displayed so that users can see the current
location on the 3-D map. When the map is clicked by the user, the system
shows the location of the mouse cursor in a list. The items in this list
can be selected and the MMHIS will query to the selected location to
display the selected video and data records. Users can zoom in, pan, and
zoom out freely using the small zooming window in the lower right corner
of the terrain map, as shown in Figure 1. On the terrain surface, the
dot shows the current query location. The viewing angles can also be
adjusted in Figure 1 by using the two small slider bars by the side of
the small map in the zooming window.

The following are additional MMHIS operations.

-   The video runs at 30 frames per second with the resolution of 640
    by 480. Data in the site data window will change dynamically with
    the motion video. The video window can be resized.

-   Data query can be conducted by (1) keying in the route number,
    direction, and milepost, or (2) dragging the slider on the slider
    bar on the bottom of the video window to a new location.

-   The data update rate can be selected through the slider bar at the
    bottom of the site data table.

-   The user can open multiple video windows simultaneously.

-   Making turns. Users can make turning movements at intersections or
    exit ramps, through user selectable turning arrows, shown in Figure
    2.

Figure 3 shows the diagram of system level design of 3-D visualization
in MMHIS. The 3-D visualization in MMHIS involves three tasks: the
database design, the database preparation, and the user interface
design. Databases offer the necessary data to render the map, and the
user interface presents the map to the user. In MMHIS several database
files are used in the 3-D rendering. The files provide elevation data,
the two-dimensional (2-D) map data, the location information, and the
intermediate rendering results. MMHIS uses custom designed database file
formats to achieve high speed data accessing. The user interface design
involves the design of three OpenGL C++ wrapper classes that facilitate
the rendering of the 3-D map. It also involves the design of a picking
algorithm that allows users to click on the 3-D map and get site data
information for the corresponding point. Zooming and panning operation
on the 3-D map is also implemented.

# Data Sources and Data Preparation For the 3-D visualization

## MMHIS 3-D Map Internal Data Formats for DEM

### General Information on USGS DEM and the Raster 2-D Map

3-D terrain data is needed for MMHIS to have a user interface of 3-D
surface. The Earth Science Information Center (ESIC) distributes digital
cartographic/geographic data files produced by the U.S. Geological
Survey (USGS) as part of the National Mapping Program. The data type
used in MMHIS is called a Digital Elevation Model (DEM), which consists
of a sampled array of elevations for a number of ground positions at
regularly spaced intervals.

In particular, the 1-degree DEM (3- x 3-arc-second data spacing)
provides coverage in 1- x 1-degree blocks. The basic elevation model is
produced by the Defense Mapping Agency (DMA), but is distributed by USGS
in the DEM data record format. In addition, MMHIS uses a 2-D map that
was scanned from a paper map as the basis for the features of the 3-D
map. Information from the 2-D map is used in the final rendering of the
3-D map. It is also used in the road digitizing process. The resolution
of this map is 1,800 pixels by 1,800 pixels. The resolution of the
terrain map is 7200 pixels by 4800 pixels.

### DEM Data File Format for MMHIS

There are 24 1-degree DEM files covering the state of Arkansas. The
files are in fixed-length ASCII format. In MMHIS, the 24 DEM files were
converted into one binary file, based on the following reasons: (1) it
is more efficient to read one file than reading 24 separate files to
render the surface, (2) the one binary file takes much less space than
the 24 separate files, due to the elimination of header information in
each of the 24 files. The DEM data presents position information in a
grid-rectangular format with approximate resolution of 70 meters for
longitudinal and latitudinal measurements. The conversion retains the
original positioning data and resolution of the 24 DEM files.

There are three different records in the converted DEM file. The first
record stores the origin of the area that is covered by the DEM. It
contains two floating-point numbers representing the longitude and the
latitude of the origin in degrees. The second record contains integers
representing the number of grid points along both the longitude and
latitude directions. The third record contains the elevations in meters
of the grid points along the profiles organized from south to north and
from west to east. The numbers are in internal integer formats.

### Other Data Files for Rendering

In addition to the DEM data file and the 2-D raster map file, the
rendering in MMHIS also uses several other data files, as follows:

-   2-D surface map file. It is a raster image file in Windows bitmap
    format based on the scanned 2-D raster map file. When the 3-D map is
    viewed from top directly down, the 3-D map changes to a 2-D map
    showing the raster image of the state. When viewed from an angle,
    this raster image drapes on top of the surface model in real-time.

-   2-D coordinate map files in Windows bitmap format. MMHIS uses these
    files to identify the longitude and latitude of any point on the 2-D
    map for picking operations. Altitude value is obtained by querying
    the converted DEM file.

-   2-D zoom surface map file in the Windows bitmap format. This file is
    similar to the 2-D surface map file, but has smaller size. It is
    used to speed up the zooming control in the MMHIS's map interface.

-   2-D zoom coordinate map files in the Windows bitmap format. These
    files are similar to the 2-D coordinate map files, but have smaller
    sizes. They are used in the zooming control in MMHIS's map
    interface.

-   Raster map information file. This file is used in a texture mapping
    process to project the 2-D map to the 3-D terrain surface.

    As the 2-D size of the raster map is smaller than the size of
    longitude and latitude in the DEM, the raster map was linearly
    stretched to match the size of the DEM at the resolution of 7,200
    longitudinal pixels and 4,800 latitudinal pixels. The colors of the
    raster map were then directly projected onto the surface of the DEM.
    The colors are saved into this raster map information file, which
    contains longitude, latitude, elevation and color of each pixel in
    the DEM. The projection principle is shown in Figure 4.

    The information in this raster map information is used for final
    screen sub-sampling and rendering. The file contains three records.
    The first record stores the origin of the area, which uses two
    floating-point numbers to represent the longitude and the latitude
    of the origin, or the lower-left corner of the DEM. The second
    record stores the total number of grid points along the longitude
    and latitude lines. The third record stores the texture mapping
    information, or the color of each pixel. The majority of the data in
    the file is used for the third record, which is an array of RGB
    values ordered along the grid lines from south to north, west to
    east. Each RGB value contains three bytes that represent the mapped
    red, green, and blue color element on that grid point.

-   Shape information file. Results of rendering of the terrain surface
    are stored in this file. As rendering data is saved in this file,
    picking and other operations can be repeatedly applied to the
    surface without re-rendering. This file does not record hidden
    surface information. OpenGL is still used to render the scene to
    show the 3-D objects' positions. The real rendering process uses
    only one light source to reduce the computation overhead, resulting
    in efficient rendering of the scene.

-   Location information file. The location information file contains
    the route number and milepost information for each grid points.
    MMHIS uses this information to decide the location of a point on the
    map. There are three records in this file. The first two records are
    the same as in the shape information file. The third record contains
    an array of integer pairs representing the route number and milepost
    of each grid points. An interactive digitizing utility was developed
    in the research to map highway routes on the 2-D raster map.

# 3-D visualization methods and their implementations

OpenGL is the industry standard software interface for high-quality 3-D
graphics applications. OpenGL is hardware-independent and available on a
variety of platforms and operating systems, including Windows NT. The
interface consists of more than 150 distinct commands that can be used
to specify the objects and operations needed to produce interactive 3-D
applications (Fosner 1996). With OpenGL, 3-D object is constructed with
a small set of geometric primitives---points, lines, and polygons. The
terrain model in MMHIS's 3-D user interface was constructed directly
with OpenGL Application Programming Interface (API). Woo, Neider, and
Davis (1997) provide a guide to OpenGL programming, which was frequently
used for this research.

## Terms in 3-D Transformation

When a 3-D object is to be viewed, the positions of both the object and
the viewer may need to be changed for proper viewing results. This
process is called coordinate transformation. Here are some important
concepts in OpenGL for coordinate transformation that were used in the
terrain visualization of MMHIS.

(1) Viewpoint is the location of the user's eyes. Its initial position
    is at the origin of the world coordinate system.

(2) The process of moving the terrain model is called model
    transformation. When viewpoint is moved, it is called view
    transformation. If both model transformation and view transformation
    occur, it is referred to as modelview transformation. However,
    OpenGL internally treats view transformation as model
    transformation. Therefore, mathematically within OpenGL, the only
    transformation is model based. Even though logically the term of
    view transformation is used, the transformation is actually achieved
    through model transformation in OpenGL. As a result, the position of
    the viewpoint is internally always at the origin of the world
    coordinate system, disregard of any type of transformations being
    conducted.

(3) Projector is a vector pointing from the center of the object being
    viewed to the viewpoint.

(4) Up vector is used to determine the orientation of the object being
    viewed. During transformation, in order to ensure that this
    orientation is preserved, a vector is used in the transformation
    calculations. This vector is called up vector, which is defined by a
    3-D position in the world coordinate system. The direction and
    length of the vector are from the origin of the world coordinate
    system to the 3-D position. However, the length of the vector is not
    used in transformation. In MMHIS, for the convenience of
    calculation, the direction of the vector is tangent to the center of
    the terrain map. Also in MMHIS, the north of the terrain map must
    always point from bottom to the top.

(5) Projection is an approach to map 3-D objects to a 2-D surface. There
    are two projection methods: perspective and parallel. In MMHIS, the
    3-D terrain map is projected to the 2-D viewing surface using
    parallel projection. Perspective projection was not chosen because
    accurate measurement of the map is needed to calculate the zoom
    factors and to conduct zooming and panning operations.

(6) A viewing volume is a portion of 3-D space that is projected to the
    2-D viewing surface. Anything outside this volume is clipped, or not
    shown in the terrain map. In parallel projection only the viewing
    volume's width and height are important to the rendered view. The
    near and far planes only need to be set to proper locations so that
    the area of the earth's surface to be shown is between these two
    planes.

(7) As parallel projection is used, as long as the user is in front of
    the terrain map, the distance between the user and the terrain map
    does not affect the visualization of the map. For the sake of
    convenience, MMHIS always uses the radius of the earth as the
    distance between the user and the map.

## View and Map Transformations in MMHIS

### Initialization of the Terrain Map: Model Transformation

In OpenGL, the initial viewpoint is always at the center of the world
coordinate system disregard of any type of transformations being
conducted. For the MMHIS 3-D terrain model, it is assumed that this
initial viewpoint is at the location of the center of the earth and the
direction of the view is facing the position with the longitude of east
90 degrees and the latitude of 0 degree (equator), shown in Figure 5
(a). At this point of view, the area of Arkansas cannot be viewed at the
initial viewpoint. Therefore, transformation of the model, area of
Arkansas, is necessary in order to let the viewpoint to have complete
view coverage of the area object.

The world coordinate system is always fixed as (*X, Y, Z*). In order to
make the terrain map viewable and also perpendicular to the user, first,
the model has to be moved, or translated away from the center of the
world coordinate system forming a local coordinate system (*x, y, z*)
with the earth's center as the local origin, shown in Figure 5 (b). In
this translation, the earth is translated back along the negative
Z-axis. For convenience, the moving distance is equal to the radius of
the earth.

In order to view the area object perpendicularly, the earth is then
rotated along the local x-axis counterclockwise when viewed at negative
x direction. The rotating angle is equal to the latitude degrees of the
point to be viewed, shown in Figure 5 (c). Afterwards, the earth is
rotated again along the local y-axis counterclockwise when viewed at
negative y direction. The rotating angle is equal to the longitude
degrees minus 90, as the projector goes from positive Z-axis to the
negative Z-axis. At this time, the position of the area of interest is
perpendicular to the Z-axis and to the viewer at the viewpoint, shown in
Figure 5 (d). Therefore, the position of the terrain map of Arkansas is
perpendicular to the user's view. However, the position of the viewpoint
and the center of the terrain map are approximately at the same
location, considering that the earth is not a perfect ball and the
terrain has varying elevations.

To view the terrain map from a distance, the model is then translated
again by the radius of the earth along negative direction of the Z-axis.
The area of interest is therefore fully viewable to the user. When MMHIS
is launched along with the terrain map utility shown in Figure 1, the
program conducts the above modelview transformations, so that user can
immediately view the complete terrain map.

### Adjustment of the Viewing Angle: View Transformation

When the user adjusts the viewing angles with the sliders in Figure 1 to
view the terrain map from a different position, the viewpoint is
logically moved. Internally, OpenGL moves the model to achieve the same
result. In this scenario, two parameters, α and β, are used to customize
the viewpoint position and projector direction, shown in Figure 6. The
angle α controls the vertical position of the viewpoint, while the angle
β controls the horizontal position of the viewpoint. The two sliders of
angle control are shown in the zooming utility at the bottom-right
corner of the terrain map in Figure 1.

To view the new terrain area defined by two parameters, α and β, the
whole model is reset to the world coordinate system. The viewpoint at
the center of the world coordinate system also becomes the center of the
earth. Based on the values of α and β, and the illustration in Figure 6,
the viewpoint is logically transformed from the center of the world
coordinate system to the selected position on the surface of the earth.
Thereafter, the complete modelview transformation sequences follow the
exact steps shown in Figure 5 (b) to Figure 5 (d).

### Modelview Transformation with the Presence of α and β

After the user selects a new viewing position with α and β, logically
setting the new viewpoint location is a process of creating a viewing
transformation matrix based on current viewpoint, a reference point
indicating the center of the map, and an up vector. The matrix of view
transformation resets the reference point of the center of the map and
the viewpoint point to the origin, as shown in Figure 5 (a). When a
typical projection matrix is used, the center of the scene therefore
maps to the center of the whole view area, or Arkansas in MMHIS. This
whole view area is referred to as viewport in OpenGL. Similarly, the
direction described by the up vector projected onto the viewing plane is
mapped to the positive y-axis so that it points upward in the viewport.
The up vector can not be parallel to the projector.

The following procedure illustrates the initial transformation for the
model similar to the step from Figure 5 (a) to Figure 5 (b) to prepare
for the additional transformation from Figure 5 (b) to Figure 5 (d).
However, Figure 5 (a) and Figure 5 (b) only show a special case of the
transformation, that is when α=0 and β=0.

The following view transformation is based on Woo, Neider, and Davis
(1997). Assume the viewpoint location is given by *EYE = *(*eyeX*,
*eyeY*, *eyeZ*)^T^; the center of scene is given by
*CENTER = *(*centerX*, *centerY*, *centerZ*)^T^; the up vector is given
by *UP = *(*upX*, *upY*, *upZ*). The projector vector is:

$\text{PJ} = \text{CENTER} - \text{EYE} = \begin{bmatrix}
\text{centerX} - \text{eyeX} \\
\text{centerY} - \text{eyeY} \\
\text{centerZ} - \text{eyeZ}
\end{bmatrix}$ (1)

The vectors are normalized as follows:

$P = \frac{\text{PJ}}{\left\| \text{PJ} \right\|}$ (2)

$UP^{'} = \frac{\text{UP}}{\left\| \text{UP} \right\|}$ (3)

Finally, let *S = P *× *UP'*, and *U=S *× *P*, forming a new local
coordinate system defined by three unit vectors, *S*, *U*, and *P.*

In OpenGL, all positions are defined with four values, x, y, z, and a
fourth value, which is normally one, forming a homogeneous coordinate.
Considering the presence of α and β, the model is rotated based on the
following matrix:

*R* = $\begin{bmatrix}
S\lbrack 0\rbrack & S\lbrack 1\rbrack & S\lbrack 2\rbrack & 0 \\
U\lbrack 0\rbrack & U\lbrack 1\rbrack & U\lbrack 2\rbrack & 0 \\
 - P\lbrack 0\rbrack & - P\lbrack 1\rbrack & - P\lbrack 2\rbrack & 0 \\
0 & 0 & 0 & 1
\end{bmatrix}$ (4)

The model is then translated based on the position of the viewpoint
*EYE = *(*eyeX*, *eyeY*, *eyeZ*)^T^:

*T* = $\begin{bmatrix}
1 & 0 & 0 & - \text{eyeX} \\
0 & 1 & 0 & - \text{eyeY} \\
0 & 0 & 1 & - \text{eyeZ} \\
0 & 0 & 0 & 1
\end{bmatrix}$ (5)

Therefore, the combinational modelview matrix factor is

$M = T \times R = \begin{bmatrix}
S\lbrack 0\rbrack & S\lbrack 1\rbrack & S\lbrack 2\rbrack & - \text{eyeX} \\
U\lbrack 0\rbrack & U\lbrack 1\rbrack & U\lbrack 2\rbrack & - \text{eyeY} \\
 - P\lbrack 0\rbrack & - P\lbrack 1\rbrack & - P\lbrack 2\rbrack & - \text{eyeZ} \\
0 & 0 & 0 & 1
\end{bmatrix}$ (6)

After the above transformation, the projector projects to the origin of
the local coordinate system. The up vector for the viewpoint is the
tangent vector of the meridian at the center of the viewing region and
pointing to the north. To be viewed correctly, the model can then be
further transformed based on the sequence from Figure 5 (b) to Figure 5
(d).

## Rendering of 2-D and 3-D Terrain Maps

After the completion of necessary modelview transformations and before
the terrain map is rendered and displayed on the screen, a process of
sub-sampling takes place to the map. As the map in memory and the map
shown on the screen window have different sizes, sub-sampling presents
the correctly sized terrain map on the screen through filtering out
unnecessary grid lines. Sub-sampling only occurs to 3-D rendering, as
sub-sampling in 2-D is automatically handled by the operating system.

### 2-D Rendering

Even though the 2-D raster image does not present any lighting effect by
itself, MMHIS uses OpenGL functions to create lighting effects for the
projected raster image perpendicular to the user. Two disk-based picking
buffers are used for this rendering. When displaying in the 2-D case, it
simply loads the raster and displays the image using Win32 API function
::StretchBlt(), which conducts the sub-sampling automatically. This
approach is shown in Figure 7.

### 3-D Rendering

3-D map rendering occurs when the viewing angles are not perpendicular
to the terrain map. In MMHIS's 3-D map rendering, there is a limitation
in using memorized 3-D commands and data in system memory, or display
lists of OpenGL. When user changes the zooming factor or the viewing
angles, the data sub-sampling has to be re-applied to the map, resulting
in the frequent updating of the display lists. This makes display lists
of OpenGL useless, as their purpose is to speed up rendering by saving
the commands the memory. Therefore, in MMHIS, a new technique was
derived to efficiently render the 3-D terrain map.

Rendering results are first saved in an internal buffer, and copied to
the screen window for display. Based on our test, the copying operation
is much faster than executing display lists by OpenGL. In addition,
MMHIS has the ability to specify a memory-mapped file kernel object in
the hard disk as its virtual memory, which cannot be achieved with
OpenGL's display lists. The size of an OpenGL display list can also
become very large when rendering a 3-D map with millions of vertexes and
triangles in system memory. The use of hard disk based virtual memory to
store the intermediate results alleviates the high physical memory
requirement by display lists of OpenGL.

MMHIS uses a triangular mesh for the whole area and calls OpenGL
rendering functions to draw the 3-D map surface. The partition for a
complete 1 × 1-degree region with no grid line filtered out is shown in
Figure 8. The actual rendering first filters out some of the grid lines
based on the size of the screen window, a sub-sampling process. For each
adjacent pair of sub-sampled profiles (lines along the latitude
direction), a triangle strip is rendered based on sub-sampled samples
(lines along the longitude direction). Both profiles and samples here
are USGS conventions for DEMs. The rendering order is from low latitude
location to high latitude location. The color of each vertex is
calculated according to the elevation of the point, the lighting
effects, and the color of the corresponding point on the 2-D map. OpenGL
smooth shading is used in the map rendering.

## Picking Operation

MMHIS allows users to click on the 3-D map and shows 3-D location
information specific to the clicked point on the map. The process to
identify the point being clicked on the 3-D map is called *picking*.
Each point on the 3-D map can be uniquely specified by giving its
longitude and latitude values. The map application allows users to click
on the map and, if the point clicked is a point on a highway section and
is registered in the database, displays a menu showing the point's
corresponding route numbers, directions, and mileposts. Picking can
happen anywhere on the map. An example of picking operation is shown in
Figure 1.

OpenGL is designed to support interactive 3-D graphic applications. Such
applications allow the user to identify objects on the screen and then
to move, modify, delete, or otherwise manipulate those objects. Since
3-D objects are drawn on a 2-D surface, picking can be difficult to
achieve. OpenGL provides a selection mechanism that automatically
records the objects drawn inside a specified region of the viewport.
This can be utilized to implement general purpose picking operation.

However, OpenGL's implementation of picking requires the program to
render the same scene twice. OpenGL establishes a data record in the
background for each object on the surface. After rendering, the name of
an object is determined when that object is selected. However, names of
all objects need to be input before the rendering in the programming
process. Therefore, OpenGL's picking implementation could not be
directly used for MMHIS, as the terrain surface in MMHIS is formed by
using possibly millions of small triangles and this naming convention
could not be directly used. Instead MMHIS uses a simplified mechanism to
achieve picking. It does not let OpenGL manage the name stack and the
background buffer. Rather, MMHIS creates two buffers for picking
operations. Instead of rendering the same scene to two different buffers
and specifying the names of the objects, MMHIS renders the main scene
only once. MMHIS renders the location information directly into the
picking buffers and does not use name stacks. *Colors* are used to
represent longitude and latitude grid numbers in its picking buffers.
When a point on the map is clicked by the user, the same point in the
two picking buffers are checked and the coordinates of the point can be
obtained by decoding the colors of the same point in the picking
buffers. This decoding process is the reverse of the rendering process.
Lighting effects are not produced when the picking buffers are rendered.
This ensures the colors of each pixel in the picking buffers are those
specified during the rendering process. Nor is anything on the screen
rendered, which ensures that the actual color depth of the display
device does not affect the color depth requirement of the picking
buffers. Figure 9 shows MMHIS's picking mechanism.

Because the same 3-D coordinates are used for rendering both the map and
the picking buffers, hidden surface removal is identical for the map and
the picking buffers. This ensures accurate picking operation for the 3-D
map. After the coordinates of a point on the map are obtained, the
program checks all the points in the vicinity using the data in the
location information file. The route number and milepost can be obtained
from this file if the point is registered in the digitizing process. The
route number is in fact the value of the *ID* field in the master
database table. With this information in hand, the direction information
can be queried out from this table. The returned information is used to
construct a context menu to show the position of the point. The process
of picking is shown in Figure 10.

# Conclusions

Currently, the data sources for MMHIS are from an existing highway data
vehicle and USGS. It requires less than a full-time personal to maintain
the operation of MMHIS. However, a computer network and workstations are
needed to support a multi-user environment, as digital video files have
to reside in a central location due to their large sizes. Video
decompression of MPEG-2 support is also necessary of the workstations.
This feature is becoming a standard for many new computers. 3-D
visualization hardware is recommended for the workstations, but not
required. 3-D rendering can be conducted by the CPU, but with slower
speed than a computer can do with dedicated 3-D hardware.

MMHIS provides multimedia data viewing capabilities to highway
engineers. It allows a highway agency to examine road and roadside
structures without taking certain field trips. In addition, MMHIS in a
computer network environment can be used in communicating design and
improvement ideas among engineers and managers, and to the general
public. Future work includes the use of higher resolution data from
satellite imagery vendors and USGS.

# References

(Fosner 1996)Fosner, R. (1996). "OpenGL Without the Pain: Creating a
Reusable 3D View Class for MFC." *Microsoft Systems Journal*, November,
Miller Freeman Inc., 600 Harrison Street, San Francisco, CA 94107

(Woo, Neider, and Davis 1997)Woo, M., Neider J., and Davis T. (1997).
*OpenGL Programming Guide*, second edition, Addison-Wesley Developers
Press, <http://www.aw.com/devpress/>

(US Department of the Interior 1993)United States Department of the
Interior U.S. Geological Survey. (1993). *Digital Elevation Models Data
Users Guide*, Reston, Virginia

(Kruglinski 1997)Kruglinski, D. J. (1997)*Inside Visual C++*, fourth
edition, Microsoft Press, A Division of Microsoft Corporation, One
Microsoft Way, Redmond, Washington 98052

(DiLascia 1997)DiLascia, P. (1997). "More Fun with MFC: DIBs, Palettes,
Subclassing, and a Gamut of Reusable Goodies", *Microsoft Systems
Journal*, January, Miller Freeman Inc., 600 Harrison Street, San
Francisco, CA 94107

Hanley, R. C. and Larsen, D. A. (1992) The Connecticut Photolog Laser
Videodisc Based Pavement Rating System. *Journal of Transportation
Engineering*, Vol 118, No. 2.

(Thompson 1995)Thompson, N. (1995). *Animation Techniques in Win32*,
Microsoft Press, A Division of Microsoft Corporation, One Microsoft Way,
Redmond, Washington 98052

Wang, K.C.P., Li, Xuyang, and Elliott, R.P. (1998) "[A]{.smallcaps}
Generic Multimedia Database for Highway Infrastructure Management",
Transportation Research Record, (in-print), Washington, D.C.

![](media/image1.png){width="4.5in"
height="9.0in"}![](media/image2.png){width="5.991666666666666in"
height="4.683333333333334in"}

FIG. 2. User Selectable Turning Movements

FIG. 3. System Level Design for MMHIS

FIG. 4. Illustration of Projection of the Raster Image onto the Terrain
Surface

FIG. 5. Modeling Transformations in MMHIS

FIG. 6. Customized Viewpoint Position and Projector

FIG. 7. Map Rendering in 2-D

FIG. 8. Partitioning with a Triangular Mesh

FIG. 9. Picking Operations

![](media/image3.wmf)

FIG. 10. Picking Algorithm