![](media/image1.png){width="2.8583333333333334in"
height="0.6416666666666667in"} Texas Transportation Institute The Texas
A&M University System

3135 TAMU

College Station, TX 77843-3135

979-845-1536

Fax: 979-845-6001

http://transops.edu

**[TECHNICAL MEMORANDUM]{.underline}**

**TO:** Harlan Miller

Federal Highway Administration

**FROM:** Kevin Balke, Ph.D. P.E.

Texas Transportation Institute

**DATE:** November 14, 2023

**SUBJECT: TASK 2.0 -- Comparison of Capabilities of Existing Evaluation
Tools**

The purpose of this technical memorandum is to summarize the activities
related to Task 2 of the Planning Analysis Tools for Operational/ITS
Evaluation Gap Study. The purpose of this task was to assess the
capabilities of various tools for evaluating and assessing the potential
benefits that could be derived from implementing ITS and other
operational improvements. The tools included in the assessment were as
follows:

-   ITS Deployment Analysis System (IDAS);

-   Dynamic Network Assignment Simulation Model for Advanced Road
    Telematics for Planning Applications (DYNASMART-P)

-   Microscopic Simulation (specifically CORSIM/TSIS and VISSIM^®^)

## Approach

It should be noted that this assessment was performed using information
obtained primarily from reviewing the User Manuals of each of the
evaluation tools and working through the example problems provided with
each software system. I also attended the DYNASMART-P training course
held at the University of Maryland in March, 2005. I also utilized the
result of a review of several papers and reports in which a more
comprehensive evaluation of the software systems was provided. The scope
of this project did not permit me to fully exercise the different tools
in many different operational scenarios and situations. Essentially, all
of my working experience with these tools came from manipulating the
example file provided with each of the tools.

## Comparison of Capabilities

Appendix A contains an assessment of the capabilities of the various
evaluation tools from the perspective of how these tools might be used
in the planning process to evaluate proposed ITS and operations-oriented
projects for inclusion in long-range transportation plans and
transportation improvement plans. The list of operational improvements
reflect the ITS and operations project currently listed on the USDOT
Joint Programs Office ITS webpage. Appendix B provides an assessment of
the analysis needs for these improvements with the various evaluation
tools. Below is a summary of the findings of these analyses.

[Long-Range Transportation Planning and Transportation Improvement
Program Planning]{.underline}

Of the three evaluation tools examined in this study, IDAS appeared to
be the best suited for analyzing improvements in the long-range planning
process, while DYNASMART-P and microscopic simulation appear to be
better suited for analyzing projects and alternatives in the
transportation improvement planning process. IDAS was designed
specifically for the purpose of providing long-range assessment of the
benefits and costs associated with implementing ITS and operational
projects in a region. IDAS allows the users to select potential packages
of deployment options and then uses a trip assignment process to assess
the benefits that can be derived from deploying the package in the
system. IDAS assesses impacts of operational improvements essentially by
factoring link capacities to reflect order of magnitude changes in
operations produced by different ITS and operations-oriented projects.
IDAS then accumulates the benefits over time for the life-cycle duration
of each of the improvement types selected by the user. The user can
define when on the planning horizon different operations improvements
can become operational and how long they will last. The objective of
long-range transportation planning is to identify what types of
transportation improvements need to be made to accommodate future
traffic growth, providing an "order of magnitude" assessment of the
potential improvements that can be generated from ITS and operational
deployments is appropriate for long-range planning.

Conversely, both DYNASMART-P and microscopic simulation models were
developed as tools to allow users to examine in detail specific traffic
situations and improvement scenarios. One feature of DYNASMART-P not
provided in the other tools is the dynamic traffic assignment process
used to distribute trips on the network. The assignment process is
dynamic because simulated vehicles can "select" new routes in response
to receiving traveler information. The user can assign different levels
of responsiveness to different groups of drivers. DYNASMART-P is
particularly useful in evaluating ITS improvements that utilize both
in-vehicle and external traveler information systems to alert drivers to
traffic situations. It can also be used to assess system performance
when some type of event, such as a work zone or traffic incident,
interrupts the normal operating conditions. DYNASMART is best suited for
assessing the impacts on a corridor-level where drivers can select
between multiple routes to reach their destination.

Microscopic simulation was designed primarily to assess the impacts and
effectiveness of different traffic control options in traffic
operations. Most microscopic simulation tools permit the user to control
traffic operations by mimicking the operations of actual traffic control
(i.e., traffic signals, ramp meters, etc) in the simulation program.
Car-following and driver behavior models are used to cause vehicles to
react to these controls in a similar fashion as drivers on the roadway.
It is called microscopic simulation because the computer program keeps
track of each individual vehicle in the system and updates its position
in the network at very short time intervals (from as little as every 0.1
second to as high as every second). Because some traffic scenarios and
situations can require the program to keep up with large amounts of
data, microscopic simulation is generally used to evaluate the
operations impacts of different traffic control strategies or
deployments at an intersection, facility or corridor-level. Generally,
both DYNASMART-P and microscopic simulation are used to examine traffic
scenarios that last for less for a day.

One downside to all of these tools is the time and effort that is often
required to set up and properly calibrate the tools so that they provide
an accurate representation of potential benefits that can be derived by
deploying an operational improvement. Even with IDAS, the user should go
through the calibration process to ensure that the tool can replicate
current traffic conditions on the existing network. This involves
outputs from the tools to measure traffic conditions. The user then
adjusts model inputs until the output generally reflects current traffic
conditions. This can often be a frustrating and time-consuming process.
Most agencies that are using tools to assist in their assessment of
projects do not have the time to adequately complete the calibration
process before the need to use the tool to assess projects and
alternatives arises.

[Performance Measures]{.underline}

[Table 1]{.underline} shows the performance measures used by each of the
tools in assigning benefits to propose improvements.

Table 1. Performance Measures Used by Evaluation Tools to Assess Impacts
of Deployment on Traffic Operations.

+-----------------------+-----------------------+-----------------------+
| **IDAS**              | **DYNASMART-P**       | **Microscopic         |
|                       |                       | Simulation ^a^**      |
+-----------------------+-----------------------+-----------------------+
| -   Vehicle Miles of  | -   Total / Average   | -   Total / Average   |
|     Travel (VMT)      |     Travel Time       |     Travel Time       |
|                       |     (Hours)           |                       |
| -   Vehicle Hours of  |                       | -   Average Total     |
|     Travel (VHT)      | -   Total / Average   |     Delay per Vehicle |
|                       |     Trip Times        |     (seconds)         |
| -   Average Speed     |     (Hours)           |                       |
|                       |                       | -   Average           |
| -   Person Hours of   | -   Total / Average   |     Standstill Time   |
|     Travel (PHT)      |     Entry Queue Time  |     per Vehicle       |
|                       |     (Hours)           |     (Seconds)         |
| -   Number of Person  |                       |                       |
|     Trips             | -   Total / Average   | -   Total \# of Stops |
|                       |     Stop Time (Hours) |                       |
| -   Number of         |                       | -   Vehicle           |
|     Accidents         | -   Total / Average   |     Throughput        |
|                       |     Trip Distance     |                       |
| -   Fatality          |                       | -   Average Total     |
|                       |                       |     Delay per Person  |
| -   Injury, and       |                       |     (seconds)         |
|                       |                       |                       |
| -   Property Damage   |                       | -   Person Throughput |
|     Only              |                       |                       |
|                       |                       | -   Queue Lengths     |
| -   Travel Time       |                       |                       |
|     Reliability       |                       | -   Emissions         |
|     (hours of         |                       |                       |
|     unexpected delay) |                       |                       |
|                       |                       |                       |
| -   Fuel Consumption  |                       |                       |
|                       |                       |                       |
| -   Vehicle Emissions |                       |                       |
+-----------------------+-----------------------+-----------------------+

^a^ Based on VISSIM, most microscopic simulation programs produce
similar measures

Each model produces generally the same type of performance measures.
These are generally measures of travel time and delay, travel demand,
and system efficiency. IDAS produces two measures that are not produced
by the other two software systems: one is the number of accidents and
the other is travel time reliability.

Each model uses a slightly different method of reporting these
performance measures. IDAS reports these measures by facility type or by
market sector (mode and/or trip purpose) for each of the alternatives
evaluated. In IDAS, benefits represent the change in the measured impact
between a control alternative and the ITS option. This change may be
positive or negative depending on the variable and the combined impact
of all the improvements being evaluated. IDAS then converts the benefit
information from daily to annual figures and assigns a monetary value
based on user-defined (or default) inputs. The sum of all benefits is
compared with the annualized cost to determine the overall benefit/cost
ratio and net benefit of the proposed improvements.

DYNASMART-P is really the only tool evaluated that can be used to assess
the impact that traveler information systems have on travel behavior.
DYNASMART-P collects network performance measures for that given
planning horizon (15-minute, peak-hour, or peak-period -- whatever has
been programmed by the user). In DYNASMART, network-level performance
statistics are collected separately for drivers with and without access
to traveler information. Performance summaries are also produced for
vehicles impacted by work zones and incidents, and for vehicles using
HOV/HOT facilities. DYNASMART can also produce link-level performance
statistics.

In addition to the measures shown in the table above, DYNASMART and many
microscopic simulation tools collect travel information for every
vehicle in the network. This vehicle trajectory information can be used
to determine how improvements affect individual trips.

[Air Quality Conformity]{.underline}

Both IDAS and most microscopic simulation models produce estimates of
fuel consumption, and vehicle emissions that could potentially be used
to assess air quality impacts of different operational improvements.
Table 2 shows the emissions measures generally produced by each
approach.

Table 2. Environmental Performance Measures Produced by Each Evaluation
Tool.

  ------------------------------- --------------- ----------------- ------------
  **Measure**                     **IDAS**        **Microscopic     
                                                  Simulation**      

                                                  **CORSIM/TSIS**   **VISSIM**

  Fuel Consumption                X               X                 X

  Evaporation Hydrocarbons (HC)                                     X

  Benezene                                                          X

  Carbon Monoxide (CO)            X               X                 X

  Carbon Dioxide (CO~2~)          X                                 X

  Hydrocarbons (HC)               X^a^            X                 X

  Non-methane Hydrocarbons (NMHC) X^a^                              X

  Non-methane Organic Gasses      X^a^                              X
  (NMOG)                                                            

  Nitrous Oxides (NO~x~)          X               X                 X

  Particulate Matter              X                                 X

  Sulfur Dioxide (SO~2~)                                            X

  Soot                                                              X

  Noise                           X^b^                              
  ------------------------------- --------------- ----------------- ------------

^a^ combined into single measure

^b^ Limited to only some improvements

Most microscopic simulation models produce vehicle emission and fuel
consumption estimates on a per link basis, while IDAS produces estimates
for either a defined evaluation area (such as a corridor) or for the
system as a whole. IDAS generates fuel consumption and vehicle emission
estimates for the life-cycle of the proposed improvement. With
microscopic simulation, emissions and fuel consumption estimates are
produced only for the evaluation period. The user would be required to
perform post-processing of these measures to produce life-cycle
estimates.

[Alternatives Analysis at the Project Level]{.underline}

Many agencies are not using any of these tools to evaluate different
project-level alternative deployments for a number of reasons including
the following:

-   Lack of knowledge and training on how to use the tools;

-   Short time windows in which to perform alternative analyses; and

-   Most operating agencies already know the type of deployment they
    want to make and just need to conduct an economic assessment of the
    benefits and costs of the alternatives.

IDAS does allow the user to directly compare different alternatives in
the same simulation run. To perform an alternatives analysis using
microscopic simulation or DYNASMART-P, the user would be required to run
two separate evaluations and then compare the results off-line. The
advantage of using these tools over IDAS, however, is that both
microscopic simulation and DYNASMART-P (if properly calibrated) would
produce results more closely in line with actual improvements.

[FTA New Starts]{.underline}

Both IDAS and DYNASMART-P allow the user to evaluate the use of transit
service in a corridor. This capability could potentially be used to
analyze the need for starting a new transit service in a corridor. Some
microscopic simulation models, such as VISSIM, can model not only bus
operations, but also light-rail and heavy-rail operations. While these
tools may not be appropriate for determining in which corridor to start
a new transit line or service, they do allow the user to quantify the
impacts of different types of transit service on traffic operations,
particularly if that service is running concurrently in the automobile
traffic in the same right-of-way.

[NEPA Studies]{.underline}

Other than fuel consumption and vehicle emissions data, IDAS is the only
tool that provides estimates of noise impacts produced by various ITS
improvements. This noise information could potentially be used in an
assessment of the environmental justice impacts of potential ITS and
operational improvements. Neither DYNASMART-P nor the microscopic
simulation models reviewed (i.e., CORSIM and VISSIM) provide a direct
analysis of noise levels produced by improvement alternatives.

[Corridor Level Analysis]{.underline}

Any of the tools can be used to evaluate projects at a corridor level,
but microscopic simulation is particularly well-suited for conducting
analyses at the corridor level. In IDAS, the user can identify a
particular portion of the network or sub-area in a region from which to
collect performance measures related to a particular improvement. In
most evaluations where DYNASMART-P and microscopic simulation are used,
only the corridor of interest is generally included in the model.

[Work Zone Impacts and Other Variable-Time Events]{.underline}

One distinct capability that DYNASMART-P and microscopic simulation have
over IDAS is that they can be used to assess traffic events that are
variable over time (i.e., events that do not have rigid starting and
stopping times). These traffic events are generally situations where the
capacity on a facility is temporarily reduced (such as in the case of a
work zone or traffic incident) or demand is temporarily increased (such
as in the case with a special event). Because DYNASMART-P and
microscopic simulation tools allow the user to alter demand and /or
capacity dynamically during the evaluation, these tools are better
suited for assessing impacts of strategies and systems for managing
traffic in work zones, incidents, and special events.

DYNASMART-P and microscopic simulation tools can also evaluate
strategies that are time-variable (i.e., strategies that are employed
when specific traffic situations or congestion levels develop on a
facility). This would include strategies such as congestion pricing,
high occupancy tolling lanes, variable toll lanes, etc. DYNASMART-P
permits that user to specifically evaluate these types of operational
strategies.

[Cost/Benefit Comparisons]{.underline}

IDAS is the only tool that provides a direct computation of benefits and
costs. The other evaluation tools require that the user post-process the
output of the model results to generate economic benefits associated
with operational improvements. The downside to IDAS is that the user
directly assigns the amount of benefit that can be derived from
implementing a particular improvement. Theoretically, it is possible for
a user to favor a particular improvement (either wittingly or
unwittingly) by artificially assigning a higher degree of improvement
(or even accepting an improvement default value) than might be
appropriate for a given situation. To combat the potential biasing of
results, it is recommended that users implement a sensitivity analysis
approach where multiple ranges of improvements are used in the
evaluation to provide a clearer insight into the amount of benefits that
can be derived from a particular strategy.

[Modal Split Implications]{.underline}

IDAS appears to be the only tool that can capture changes in modal split
that may occur as a result of making transit trips more attractive
through operational changes, such as transit signal priority, or high
occupancy tolling (HOT) strategies. As part of the modeling process,
IDAS runs a mode-choice model that distributes trips to transit. If an
operation strategy improves transit operations (i.e., makes it more
attractive to some trips), more trips would be assigned to transit
vehicles. With DYNASMART-P and microscopic simulation, a fixed portion
(or percentage) of the overall volume in the corridor is generally
assigned to a particular mode. The user must change this portion in the
input file in order to reflect mode choice changes that may result from
a particular ITS or operational strategy.

[Changes in Travel Demand]{.underline}

IDAS also has the capabilities of modeling temporal and induced/foregone
changes in travel demands. IDAS contains a temporal choice model that
allows trips to be redistributed to outside (or possibly into) the
peak-hour to model how the peak period spreads (or contracts if
operational improvements are successful) in response to congestion. IDAS
also models induced and foregone demand that can occur as a result of
making travel time improvements. To have the similar effect in the other
tools, the user would have to alter the basic input volumes used in the
models.

[Changes in System Capacity]{.underline}

One advantage microscopic simulation and DYNASMART-P have over IDAS is
that it is relatively easy for the user to reflect changes in system
capacity caused by adding new links or lanes to a link. With IDAS, a
user is required to go back to the original travel demand model to add
new links and nodes to the basic evaluation network.

DYNASMART-P and microscopic simulation can also evaluate Transportation
Systems Management (TSM) -type capacity enhancements (adding left-turn
bays, and turning lanes, etc.). Often, simple TSM-type improvements are
made in conjunction with deploying some types of operational
improvements or to get full benefit of operational improvements. IDAS
does not easily permit the users to assess the benefits of these types
of small capacity enhancements.

[Strategic Visioning]{.underline}

None of the tools are particularly ideal for supporting the development
of strategic visioning of ITS and operational improvement deployments.
Strategic visioning involves examining how various ITS and operational
deployment can be combined and their deployments sequenced so that
systems (and their benefits) build upon one another to maximize the
return on investment in a region. An example of strategic visioning
would be the development and deployment of a freeway surveillance and
control system that supports not only incident management, but also ramp
metering, motorists information systems, managed lanes applications, and
so on. Strategic visioning is intended to prevent agencies from
deploying technologies which they cannot support, maintain, or operate
given their level of deployment, staffing, training, and funding.

Ideally, a tool that supports strategic visioning would be one that
assists agency and regional decisions makers as to what ITS and
operational improvements are appropriate and when they should be
deployed. Agencies would start by entering their current ITS and
operational deployments. Agencies would also need to identify what types
of improvements are logical and practical given the composition of their
region. With this background information, this decision support system
would examine project traffic pattern changes over time and determine
when in the planning horizon certain ITS and operational deployments
need to occur in order to maximize their return on investment. It would
not only determine what systems are needed at what point in the future,
but it would also lay out for the decision-makers a strategic path for
deploying the technologies in the right corridors. Issues such as
regional balance and environmental justice could be incorporated into
the tool.