ARL-TN-0797 ● NOV 2016

US Army Research Laboratory

Developing a Value of Information (VoI)-

Enabled System from Collection to Analysis

by Mark R Mittrick, John T Richardson, Alex Vertlieb, and 

Timothy P Hanratty

Approved for public release; distribution is unlimited.
ii

NOTICES

Disclaimers

The findings in this report are not to be construed as an official Department of the 

Army position unless so designated by other authorized documents.

Citation of manufacturer’s or trade names does not constitute an official 

endorsement or approval of the use thereof.

Destroy this report when it is no longer needed. Do not return it to the originator.
ARL-TN-0797 ● NOV 2016

US Army Research Laboratory

Developing a Value of Information (VoI)-

Enabled System from Collection to Analysis

by Mark R Mittrick, John T Richardson, and

Timothy P Hanratty

Computational and Information Sciences Directorate, ARL

Alex Vertlieb

CUBRC, Buffalo, NY

Approved for public release; distribution is unlimited.
ii

REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the 

data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the 

burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. 

Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid 

OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

November 2016

2. REPORT TYPE

Technical Note

3. DATES COVERED (From - To)

October 2015–September 2016

4. TITLE AND SUBTITLE

Developing a Value of Information (VoI)-Enabled System from Collection to 

Analysis

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S)

Mark R Mittrick, John T Richardson, Alex Vertlieb, and Timothy P Hanratty

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

US Army Research Laboratory

ATTN: RDRL-CII-T

Aberdeen Proving Ground, MD 21005-5067

8. PERFORMING ORGANIZATION REPORT NUMBER

ARL-TN-0797

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S)

11. SPONSOR/MONITOR'S REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release; distribution is unlimited.

13. SUPPLEMENTARY NOTES

14. ABSTRACT

Developing and sustaining situational awareness is an enduring challenge for today’s Warfighter. Situational awareness, or 

knowledge of the immediate present environment, is a daunting task considering the plethora of data sources available. The 

US Army Research Laboratory has confronted this challenge by researching and developing the Value of Information (VoI) 

metric. In this report, we discuss a prototype system that pushes VoI beyond simulations and into the field environment where

further research can be conducted.

15. SUBJECT TERMS

DSPro, VoI, Value of Information, Android, smartphone, information dissemination, visual analytic

16. SECURITY CLASSIFICATION OF:

17. LIMITATION

 OF

 ABSTRACT

UU

18. NUMBER

 OF 

 PAGES

28

19a. NAME OF RESPONSIBLE PERSON

Mark R Mittrick

a. REPORT

Unclassified

b. ABSTRACT

Unclassified

c. THIS PAGE

Unclassified

19b. TELEPHONE NUMBER (Include area code)

410-278-4148

Standard Form 298 (Rev. 8/98)

Prescribed by ANSI Std. Z39.18
Approved for public release; distribution is unlimited.

iii

Contents

List of Figures iv

List of Tables iv

1. Introduction 1

2. Developmental Work 2

2.1 Spot Report App 2

2.2 Checkpoint Report App 6

2.3 Common Features 13

2.4 DSPro Integration 15

3. Contour 15

4. Conclusions 19

5. References 20

List of Symbols, Abbrevations, and Acronyms 21

Distribution List 22
Approved for public release; distribution is unlimited.

iv

List of Figures

Fig. 1 Spot report main screen .........................................................................2

Fig. 2 Smartphone app workflow.....................................................................3

Fig. 3 VoI screen ..............................................................................................3

Fig. 4 Checkpoint report main screen ..............................................................7

Fig. 5 Checkpoint report main screen 2 ...........................................................7

Fig. 6 VoI screen 2 ...........................................................................................8

Fig. 7 Data JSON field ...................................................................................14

Fig. 8 Contour; no VoI...................................................................................17

Fig. 9 Contour; VoI........................................................................................18

List of Tables

Table 1 Data elements and values.......................................................................8
Approved for public release; distribution is unlimited.

1

1. Introduction

Developing and sustaining situational awareness is an enduring challenge for 

today’s Warfighter.

1 Situational awareness, or knowledge of the immediate present 

environment, is a daunting task considering the plethora of data sources available. 

In particular, an intelligence analyst must be able to search through an enormous 

haystack in a timely manner to find the few valuable needles. The US Army 

Research Laboratory (ARL) has confronted this challenge by researching and 

developing the Value of Information (VoI) metric.

2

The VoI metric uses a fuzzy logic-based model that converts the subjective 

experience of subject matter experts (SMEs) into a range of scores. Specifically, 

given an evaluation of the source reliability and information content of an 

intelligence report, the ARL model will produce a VoI score that approximates the 

SME process for determining value. Hypothetically, this score will enable an 

intelligence analyst to identify the most valuable information necessary to sustain 

situational awareness. Simulations conducted by ARL have demonstrated that the 

VoI metric can increase the situational awareness of an intelligence analyst during 

a finite time period.

3

In this report, we discuss a prototype system that pushes VoI beyond simulations 

and into the field environment where further research can be conducted. In general, 

the system consists of VoI-enhanced 1) collection tool, 2) dissemination service, 

and 3) visual analytic. The collection tool developed by ARL will be embedded 

with a Soldier that will enable them to create and disseminate VoI-rated reports. In 

this scenario, the Soldier is the data collector and SME. Their subjective evaluation 

of the of collected intelligence initiates the VoI rating that will persist all the way 

to the analyst. The dissemination service leveraged in this prototype is DSPro, 

software that was developed by the Institute for Human and Machine Cognition 

(IHMC) in collaboration with ARL. Finally, the visual analytic for this prototype 

is an enhanced version of Contour (a relationship discovery tool developed by 

Decisive Analytics Corporation). The ARL enhancements to Contour use the VoI 

rating of the source data as a means of filtering the relationship diagram. The 

development and integration of each component will be discussed in detail in the 

proceeding sections.

In conclusion, the goal of this research is to develop a prototype system that will 

allow ARL to evaluate VoI in a field environment. Simulations already suggest that 

VoI is a candidate for helping sustain situational awareness, but field experiments 

are necessary for further evidence. This prototype provides VoI enhancement at all 

stages from collector to analyst and can be used as a field experimentation platform.
Approved for public release; distribution is unlimited.

2

2. Developmental Work

This section outlines the software development necessary for building the 

prototype. Development for this prototype included the creation of 2 Android 

smartphone applications (apps) and the enhancement of an existing tool (Contour). 

Prior work with Android smartphone apps was leveraged to quickly develop both 

apps.

4 This allowed for more time to test and refine the prototypes, which resulted 

in a superior end product.

Enhancing Contour with VoI functionality presented a unique challenge since the 

system was largely a black box with no source code available. With help from one 

of the original developers (Scott Martin, Decisive Analytics Corporation), VoI and 

Contour were successfully integrated.

2.1 Spot Report App

The Spot Report app was developed to report observations from the field that could 

have an immediate and significant effect on current planning and operations. The 

Soldier using the app will also subjectively assign a VoI score (Fig. 1) to the report. 

Finally, the report is disseminated to analysts for further evaluation. Please see the 

smartphone app workflow diagram (Fig. 2) for the complete process.

Fig. 1 Spot report main screen
Approved for public release; distribution is unlimited.

3

Fig. 2 Smartphone app workflow

To begin, the Soldier launches the app and is presented with the main screen 

(Fig. 3), where report content is recorded. 

Fig. 3 VoI screen

The app was designed with simplicity and ease of use in mind, consequently some 

data are automatically generated using the smartphone’s built-in capabilities. These 

fields (Date, Time, and Location) are intentionally greyed out to signify that they 

cannot be manually edited. The remaining fields (Source, Report, Picture, Audio, 

and VoI) are editable and the Soldier may complete any combination of fields 

before submitting the report. 

The following is an example of output disseminated from the Spot Report app 

(DSPro required Extensible Markup Language [XML] format).
Approved for public release; distribution is unlimited.

4
Approved for public release; distribution is unlimited.

5
Approved for public release; distribution is unlimited.

6

Examining the XML reveals the Spot Report content is located in the “Application 

Metadata” field in JavaScript Object Notation (JSON) format. This format was 

selected for compatibility with other tools under development within ARL. The 

corresponding JSON schema for parsing the spot report content is defined in the 

following. 

2.2 Checkpoint Report App

The Checkpoint app was developed to report observations from processing people 

through checkpoints. The Soldier using the app will also subjectively assign a VoI 

score (Fig. 4) to the report. Finally, the report is disseminated to analysts for further 

evaluation. The checkpoint workflow is the same as the spot report workflow and 

can be found in Section 2.1.
Approved for public release; distribution is unlimited.

7

Fig. 4 Checkpoint report main screen

To begin, the Soldier launches the app and is presented with the main screen 

(Figs. 5 and 6), where report content is recorded. 

Fig. 5 Checkpoint report main screen 2
Approved for public release; distribution is unlimited.

8

Fig. 6 VoI screen 2

This app was designed with similar design principles as the Spot Report app, so 

again, simplicity and ease of use were the two main goals. Similar to the Spot 

Report app, some of the data is automatically generated using the smartphones 

built-in capabilities. These fields (Data, Time, and Location) are intentionally 

greyed out to signify that they can’t be manually edited. The remaining fields 

(Name, Nationality, Criminal Record, Education, Employment, Religion, Address, 

Tribal Affiliation, Gender, and Age) are selectable and the Soldier may complete 

any combination of fields before submitting the report. 

Table 1 shows the data elements in the Checkpoint app along with the range of 

values.

Table 1 Data elements and values

Data element name Possible values

Name Suspect’s name

Nationality Afghan, Muslim,

Criminal record Has record, no record

Education High, medium, low

Employment Cleric, laborer, professional, retired, unemployed

Religion High, medium, low

Address

Times Square Village, Viet Nam Village, Vertol

Village, Hanover Village, Cook Corner Village, Gredge town,

Utes Village, Horizon Village

Tribal affiliation Baloch, Hazara, Pashtu, Tajik

Gender Male, female

Age 20, 25, 30, 35, 40, 45, 50, 55
Approved for public release; distribution is unlimited.

9

The following is an example of output disseminated from the Checkpoint app 

(DSPro required XML format).
Approved for public release; distribution is unlimited.

10
Approved for public release; distribution is unlimited.

11
Approved for public release; distribution is unlimited.

12

Examining the XML reveals the Checkpoint content is located in the “Application 

Metadata” field in JSON format. This format was selected for compatibility with 

other tools under development within ARL. The corresponding JSON schema for 

parsing the Checkpoint message is in the following.
Approved for public release; distribution is unlimited.

13

2.3 Common Features

The Spot Report and Checkpoint Report apps share numerous features that 

streamline the Warfighter’s report generation process:

1) Time and GPS coordinates included in the reports are automatically 

gathered from the smartphone. 
Approved for public release; distribution is unlimited.

14

2) Ability to include a photograph in the report. The photo can be taken during 

report creation using the smartphone camera or selected from the phone’s 

gallery. This feature allows the analyst to include a visual depiction of the 

report content. 

3) The apps function regardless of network connectivity. Reports will be 

recorded locally within the device and disseminated whenever a connection 

is established. 

4) Audio may be recorded from within the apps. This allows the Soldier to 

record and disseminate recorded conversations providing the analyst

maximum context. There may be a communication barrier during 

checkpoint operations, but recording conversations prevents information 

from being lost in the chain from collection to analyst. An analyst may have 

expert translators available, ready to evaluate audio recordings of interest. 

When the audio feature is enabled, the corresponding JSON code (Fig. 7) is 

used to facilitate inclusion of both the photo and audio messages in the data 

field.

5) The apps use the phone’s built-in speech-to-text capabilities. The Soldier is

not forced to attempt to type on the phone’s touch keyboard to input data. 

There is an optional stylus that may be used to navigate the apps and record 

text. 

6) Both apps include a “one-touch” VoI recording option. Touching the VoI 

grid on the app sets the Source Reliability and Information Content to the 

corresponding value. Sliders for Source Reliability and Information Content 

are also available in case isolated control of each field is desired. 

The set of streamlined usability features across the apps is designed to allow the 

Soldier to focus on the mission and avoid struggling with the technology. 

Fig. 7 Data JSON field
Approved for public release; distribution is unlimited.

15

2.4 DSPro Integration

This prototype uses DSPro as its dissemination service. DSPro is a technology 

developed by the IHMC* that efficiently and proactively disseminates information

to consumers by means of replication and forwarding policies. The proactive 

behavior is the result of the integration of policies and a distributed-learning 

algorithm that takes into account the history of previously requested information, 

along with the characteristics of the collectors and the mission. As new information 

becomes available, DSPro matches it against the mission profile and pushes 

relevant information to the consumers. Information that is selected to be pushed is 

sorted based on the predicted time to use and the confidence value of the 

prediction.†

In FY16, ARL’s Multilingual and Analysis and Battlefield Information Processing 

groups collaborated with the IHMC to expand and enhance DSPro. As part of that 

project, DSPro was integrated into the Checkpoint and Spot Report apps. In 

addition, the VoI metric along with its components (Information Content and 

Source Reliability) were integrated into DSPro. It’s important to note that DSPro 

takes the VoI score it receives from the smartphone apps and combines it with 

additional data from other sources to form its own VoI score.

3. Contour

Contour is a software tool developed by Decisive Analytics Corporation (DAC) in 

collaboration with ARL. It allows an intelligence analyst to collect, search, and 

analyze large amounts of data. It automatically processes and discovers 

relationships within the ingested data, which allows the analyst to save time and 

effort deriving actionable intelligence. Contour’s workflow is based on the open 

source algorithms in Framenet,

‡ Gate, and Annie.§ These algorithms allow it to 

generate the relationship diagrams and social network maps. 

As mentioned previously, modifying Contour was challenging, as the software was 

a “black box”. With some insight from DAC, ARL was able to incorporate VoI into 

Contour by overriding the Date field, since date filtering was not of interest in this 

iteration. Specifically, the year portion of the date field was overridden to represent 

the VoI score of the data. For example, the year 2000 would represent a VoI score 

of “0” and the year 2010 would represent a VoI score of “10”. With this change 

 * The Institute for Human and Machine Cognition is a not-for-profit institute of the Florida University 

System that frequently collaborates with ARL. http://www.ihmc.us/.

† Getting Started with DisServicePro, Giacomo Benincasa/Niranjan Suri, 2015.

‡ https://framenet.icsi.berkeley.edu/fndrupal/.

§ https://gate.ac.uk/.
Approved for public release; distribution is unlimited.

16

implemented, the data set could be modified to include the VoI score for use with 

Contour. 

Figure 8, shows the original Contour social network map for the Ali Baba Data Set. 

In contrast, Fig. 9 shows the VoI-enhanced social network map for the Ali Baba 

Data Set. In this example, the VoI filter is set to 5 or greater, resulting in the lower 

VoI relationships being greyed out. With VoI enhancements enabled, an analyst 

may quickly triage the information and identify the most important people, 

organizations, and locations. The links in the graph are extracted from multiple 

reports, but ARL enhanced the link thickness algorithm to reflect the highest VoI 

score from amongst those reports. 
17

Approved for public release; distribution is unlimited.

Fig. 8 Contour; no VoI
18

Approved for public release; distribution is unlimited.

Fig. 9 Contour; VoI
Approved for public release; distribution is unlimited.

19

Another issue encountered with Contour was entity recognition. To illustrate, 

Contour does not recognize Arabic names. This issue was identified when 

Contour’s social network graph was compared to ground truth and found to be 

incomplete. To correct this, the Contour gazetteer or “whitelist” was modified to 

include all of the unique names and locations in the dataset. This method resolved 

most of the name issues; however, a few names had to be modified to be recognized 

by Contour. It is believed that the upcoming version of Contour will fix this 

problem.

Testing the enhanced version of Contour required a data set with a known ground 

truth. Consequently, the Ali Baba Data Set was selected. ARL is familiar with the 

Ali Baba Data Set from past experience.

5 The data set contains 625 messages in 

individual text (.txt) files, which contain 1 or 2 short sentences. As mentioned 

previously, the data set had to be manipulated to be compatible with Contour. This 

entailed assigning the 625 documents a VoI score (by overriding the date field) and 

fixing formatting issues. ARL expertise with the dataset allowed for learned 

application of VoI scores. These modifications allowed the team to ingest the data 

set into Contour and successfully exercise the resulting relationship graph with VoI 

filters.

4. Conclusions

ARL has developed a prototype system that incorporates VoI from collection to 

analysis. This system is a candidate platform for field experiments to evaluate the 

benefits of VoI.

ARL’s Checkpoint and Spot Report apps allow Soldiers, as data collectors, to send 

VoI-rated data, derived from their subject matter expertise, from the field back to 

the analyst. The data traverses the DSPro network, which may add additional VoIbased filtering, before being analyzed by VoI-enhanced tools, such as the 

augmented version of Contour.

Simulations suggest that VoI-rated data aid the analyst in processing the most 

valuable intelligence first, which is an important component sustaining situational 

awareness. This prototype will allow further investigation of the benefit of VoI in 

a field environment. If future field exercises are successful, this would be the first 

step in developing a system for aiding the analyst to rapidly discover the most 

valuable information being collected real-time in the field. 
Approved for public release; distribution is unlimited.

20

5. References

1. Army warfighting challenges. Army Capabilities Integration Center; Fort 

Eustis (VA): c2004–2016 [accessed 2015 June 29]. http://www.arcic.army

.mil/Initiatives/ArmyWarfightingChallenges.

2. Hanratty T, Newcomb A, Hammell R, Richardson J, Mittrick M. A Fuzzybased approach to support decision making in complex military environments. 

International Journal of Intelligent Information Technologies. 2016;12(1): 1–

30.

3. Richardson J, Mittrick M, Hanratty T. Modeling the impact of value of 

information on situational awareness using C3TRACE. Aberdeen Proving 

Ground (MD): Army Research Laboratory (US); 2016 May. Report No.: ARLTR-7684.

4. Mittrick M. Smartphone applications enabling global graph exploitation and 

proactive dissemination service (DSPro) integration (revised fiscal year 2015).

Aberdeen Proving Ground (MD): Army Research Laboratory (US); 2015 Sep.

Report No.: ARL-TN-0696.

5. Mittrick M, Roy H, Kase S, Bowman E. Refinement of the Ali Baba data set.

Aberdeen Proving Ground (MD): Army Research Laboratory (US); 2012 Mar.

Report No.: ARL-TN-0476.
Approved for public release; distribution is unlimited.

21

List of Symbols, Abbrevations, and Acronyms

app application

ARL US Army Research Laboratory

DAC Decisive Analytics Corporation

GPS global positioning system

IHMC Institute for Human and Machine Cognition

JSON JavaScript Object Notation

SME subject matter expert

VoI Value of Information

XML Extensible Markup Language
22

1 DEFENSE TECHNICAL

(PDF) INFORMATION CTR

DTIC OCA

2 DIRECTOR

(PDF) US ARMY RESEARCH LAB

RDRL CIO L

IMAL HRA MAIL & RECORDS

MGMT

1 GOVT PRINTG OFC

(PDF) A MALHOTRA

3 DIR USARL

(PDF) RDRL CII T

M MITTRICK

T HANRATTY

J RICHARDSON