# The uncertain reality of underground assets

Nadia Boukhelifa, David Duke

► To cite this version:

Nadia Boukhelifa, David Duke. The uncertain reality of underground assets. ISPRS/CA/DGfK Joint Workshop on Visualization and Exploration of Geospatial Data, International Society for Photogrammetry and Remote Sensing, 2001, NA, France. hal-01607401

HAL Id: hal-01607401

https://hal.science/hal-01607401

Submitted on 4 Jun 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
THE UNCERTAIN REALITY OF UNDERGROUND ASSETS

N. Boukhelifa and D. J. Duke

School of Computing, University of Leeds, UK

**KEY WORDS:** Geospatial data, uncertainty visualization, decision making.

**ABSTRACT:**

Visualization may be a potent looking-glass for peering into both physical and abstract worlds, but our view is always distorted by inaccuracies in measurements, incompleteness of knowledge, and ambiguities in interpretation. To the visualizer, uncertainty is a particular problem. Depiction depends on the placement of marks on paper or screen, and the placement of such a mark can, implicitly, convey confidence that this feature is in this place rather than another, or even that the existence of the feature is accepted and agreed. In this paper, we use the term *uncertainty visualization* to refer to the problem of acquiring, modelling and representing data while accounting explicitly for the uncertainties that encompass it. We provide an overview of this growing area, and discuss methods of systematically incorporating knowledge of uncertainty in a geovisualization application. We report on ongoing research in a major industrial application to visualize buried assets while reflecting the uncertainty of our knowledge.

**1 INTRODUCTION**

Within the VISTA project<fnref n="1" />, we are working within a consortium of private companies providing utility services (gas, electricity, sewerage, water, telecoms, etc), civil engineering companies sub-contracted to repair and replace these assets, and government and industry bodies concerned with regulations and the effective management of these services. Changes in surveying and information management technologies over the last two centuries, compounded by major organizational changes within the utility industries, mean that data about buried assets originates from a variety of sources. Current maps used in fieldwork and planning do not usually reflect either the quality of the underlying data or its provenance, and this has a practical impact on the time taken to carry out street works, and on the likelihood of 'colateral damage'. VISTA aims to reduce street work by developing an integrated data model for buried assets, and critically for this context, supporting this integration with visualization techniques that adequately reflect the inherent uncertainty of this data.

In this paper, we review the various definitions and taxonomies related to the concept of uncertainty (section 2). There are multiple challenges pertinent to uncertainty visualization. These are summarized in section 3. We then concentrate on advances made in geovisualization towards better visual tools to communicate uncertainty information, and we highlight areas which need further investigation (section 4). Finally, we close this paper by considering evidence on whether inclusion of uncertainty information in representation is helpful and whether it improves decision making (section 6).

**2 DEFINITIONS AND TAXONOMIES**

Various definitions of uncertainty can be found in the literature which reflects the range of concerns that come under the general term. Often the relationship between 'accuracy' and 'uncertainty' is debatable. For example, inaccuracy can be used as the general term and uncertainty in more specific cases when the inaccuracy is not known (Hunter and Goodchild, 1993). Here, when inaccuracy is known objectively it can be expressed as error. Other researchers consider error part of uncertainty, where error is defined as the discrepancy between a given value and its modeled

or simulated value (Pang, 2001). Others refer to the good aspects of uncertainty as those that are concerned with reporting accuracy, and the negative aspects of uncertainty as those dealing with error (Buttenfield, 1994). There is also the view that distinguishes between uncertainty that is inherent in the phenomena, called indeterminacy, and uncertainty in assertions of that phenomenon such as the uncertainty in the measurement function (Plewe, 2002). Despite the different notions of uncertainty and the varied usage of this term, there is the general agreement that *uncertainty* is understood as a composition of multiple concepts such as error and accuracy.

Uncertainty and data quality have a close link and the two terms are sometimes used synonymously. In cartography, for instance, there is a strong tradition of attention to data quality (MacEachren, 1992). Buttenfield describes a framework to visualize data quality components in which she produced a matrix to describe all possible spatial data quality configurations, and each cell provides information on the design strategy for that case (Buttenfield, 1991, Buttenfield, 1994). In terms of the design quality, she provide three cases: the first type corresponds to the design strategies accepted by cartographers that are already tested. The second category refers to speculative strategies, i.e. strategies which are not yet accepted as conventions. The third category refers to cases where the graphical design is problematic, either because of graphical limitations or perceptual constraints.

This close link between uncertainty and data quality means that the uncertainty measures in Geographical Information Systems (GIS) derive directly from the data quality components, such as those specified in the Spatial Data Transfer Standards (STD, 2007). SDTS were invented to facilitate data exchange between different GISystems and to minimise the loss of information during the exchange by embedding meta information such as on data quality in the transfer of geo-spatial data sets. SDTS specify five categories of data quality:

* Lineage: provides information on sources, update activity with dates, and processing steps that have transformed the data (e.g. a report on derivation or compilation methods).

* Positional accuracy: information on how closely coordinate values of map features match their true value. Based on

<footnote n="1">¹VISTA: http://www.vistadtiproject.org/</footnote>
reference to latitude and longitude or another external coor-
dinate reference system using any several means to deduc-
tively estimate or rigourously test accuracy (e.g. statement
of horizontal or vertical accuracy).

* Attribute accuracy: information on the error in the values of attribute data elements included in the transfer. The error may be based on deductive estimates or actual tests (e.g. the level of misclassification of defined areas on a land cover map).

* Logical consistency: an indication of the graphic quality and the topological integrity of a digital map (e.g. a report on problems with the graphic connectivity and closure such as overshoots or gaps for a parcel or a soil map). In other words, this refers to a data structure whose topology makes sense (MacEachren, 1992).

* Completeness: information about the selection criteria for the inclusion of map features, minimum thresholds in map compilation (minimum area or width), and the exhaustiveness of features mapped (e.g. minimum mapping unit sizes for soil mappings, expected percentage of manholes mapped from aerial photography relative to the number of manholes that actually exist). In other words, this refers to comprehensive data and systematic ways of dealing with missing values (MacEachren, 1992).

The applicability of data quality components to other types of
data or for other purposes than data transfer is an interesting is-
sue. The difference between data quality and uncertainty is high-
lighted by MacEachren who argues for broadening the scope of
data quality towards the much wider and encapsulating domain
of uncertainty. In particular, he comments on spatial and at-
tribute data quality and variability (where variability implies un-
certainty): "Our level of uncertainty about map locations will be a
function not only of the quality values, but of variance around the
mean values we typically use to represent the unit, and of spatial
variability across the unit." (MacEachren, 1992).

There is also a more general view on uncertainty than the specific
data quality perspective given above. For example, a more exten-
sive list of sources of uncertainty, which includes both quantita-
tive and qualitative aspects, is provided below (Thomson et al.,
2005).

* Error: the discrepancy between a given value and its true value. Error has various components; value, space, time, consistency and completeness.

* Imprecision: related to the resolution of a value compared to the needed resolution (the intervals when sampling).

* Accuracy: the size of interval a value lies in (both measurement accuracy and class assignment accuracy).

* Lineage, source of data (provenance), age, method of derivation, transformations.

* Subjectivity: the degree of subjective influence in the data.

* Non-specificity: the lack of distinctions for objects (e.g. an attribute value is known to be one of several alternatives but not which one).

* Noise: undesired background influence.

These sources of uncertainty can be grouped into two main cat-
egories: quantitative (for example as a result of inaccuracies in
measurements) and qualitative (e.g. credibility of source, age or
currency of information). Sometimes multiple aspects of uncer-
tainty are combined to describe other types of uncertainty. For
example, validity encompasses both the accuracy of data and the
procedures applied to the data; and data quality is a more general
concept that includes data validity and data lineage (Pang, 2001).
Another aspect of uncertainty is to do with the notion of relative
and absolute uncertainty information, e.g. see (Collins and Penn,
2006). They describe relative uncertainty in the context of multi-
lingual chat visualization, where uncertainty is characterized by
alternative paths through the lattice and confidence nodes which
are denoted using colour. All these concepts help us define our
confidence in the data, and should play a role in the decision mak-
ing process.

We can conclude that GIS has contributed greatly to the formal-
ization and conceptualization of uncertainty (as well as providing
logical and physical data structures and file formats) under the
specialized area of data quality. However, the integration of un-
certainty information in the raw data and in the visual display still
needs further work. Perhaps this is because SDTS are designed
for data transfer and not for the direct use and visualization since
the data quality components are in report format. However, it has
become a standard practice for organisations to build and main-
tain separate databases for meta data containing information on
data quality and other characteristics, but this information is not
necessarily shared as it could be sensitive information.

3 THE RESEARCH CHALLENGES

Uncertainty is a major challenge for visualization that is now
widely accepted (Johnson, 2004). What is not yet agreed is how
to produce a coherent representation of data and the associated
uncertainty. We envisage three major challenges in uncertainty
visualization which are concerned with how uncertainty is pro-
vided, depicted and used.

First, one might ask whether meta data about uncertainty, such
as its provenance, is inherently any different from the other data
that are processed in visualization, i.e. whether uncertainty is
simply another dependent variable. There is the issue of salience
of uncertainty information over the base data. Uncertainty infor-
mation, in relation to raw data, can be provided in three different
formats:

* Secondary data (or meta-data) to calibrate the display but does not need immediate user attention. Typically this might be in the form of dependent variables.

* Normal data, typically this might include independent vari-
ables such as lineage and estimates on completeness.

* Primary (high-priority) data such as risk, i.e. information considered important for decision making and needs to be accessed immediately as pointers to areas in need of immediate attention.

Second, there is the visualization challenge of finding a repre-
sentation that is compatible with the nature of uncertainty infor-
mation, whether it is primary data that needs to be immediately
attended to (e.g. using pre-attentive symbology), normal data (de-
picted using the same visualization process for the raw data or
secondary data which then requires low-key visualizations that
are less distracting (e.g. peripheral visualization). See section 4 for examples of depiction techniques.

Third, the assessment of how uncertainty information is used (e.g. to support decision making) can inform the choice of how uncertainty information should be provided and depicted. Further information on evaluation and analysis of uncertainty visualization can be found in section 6.

Related literature to uncertainty in GIS details specific challenges, for example (MacEachren et al., 2005). While their review is focused on geospatial uncertainty information for decision making, the comments and findings can be extended to the visualization of uncertainty information of any kind. They conclude the review with seven major challenges facing uncertainty visualization which require multidisciplinary effort. Ranked from conceptual to more practical, there are seven major challenges:

* Clarification of concept and contexts: understanding uncertainty within its problem domains and paying more attention to abstract types of uncertainty such as consistency and completeness.

* Establishing the rapport: how uncertainty information influences analysis, and decision making and its outcomes.

* Evaluating the synergy between uncertainty visualization and exploratory analysis. An example of this synergy would be to allow the user to interact with uncertainty to gain insight to the data.

* Better methods to capture and encode uncertainty information, whether this is numerical data from measurements or qualitative data from subjective human judgement.

* Depiction of multiple forms of uncertainties, whether this is associated with spatial, temporal or attribute data. This has to be done without overshadowing the data of interest. Various methods have been suggested in the literature which tend to take either a direct or indirect approach.

* Design of methods and tools to interact with the visual representation of uncertainty (including user control over its representation), and the need to map traditional GIS analysis operations to interactions in the visualization system, e.g. direct manipulation.

* Comprehensive evaluation of uncertainty visualization, to include assessment of usability and utility of its capture, representation and interaction methods and tools.

## 4 DEPICTING UNCERTAINTY

There is a mismatch between efforts to conceptualize and to represent uncertainty (MacEachren et al., 2005). For example, uncertainty encompasses multiple components, yet visualization efforts tend to single out the individual components such as the spatial uncertainty. In addition, whereas some visualization taxonomies, e.g. (Tory and Moller, 2004), include 3D data there is a lack of representation of error and uncertainty in 3D visualization in particular (Johnson and Sanderson, 2003, Johnson, 2004). It is more common to see, for example, error bars in 2D graphs than it is to see some sort of uncertainty visualization for surfaces or isosurfaces (although there is recent work, e.g. (Grigoryan and Rheingans, 2002)).

Different types of uncertainty necessitate different methods of depiction. For example, bounded uncertainty (Olston and Machinlay, 2002) requires different visualization techniques than, say, statistical forms of uncertainty where error bars are more appropriate. Current approaches to uncertainty visualization tend to regard uncertainty from a conceptual view as either an inclusion or exclusion problem, and from a representational view as metadata which can be visualized directly or indirectly:

* Inclusive and exclusive approaches: uncertainty attributes act as data quality filters; e.g. only data that is certain is shown.

* Direct and indirect representation: uncertainty information can be represented in two ways; by altering the appearance of the underlying data using, for example, different rendering techniques (such as Non-Photo-realistic Rendering NPR (Stephenson et al., 2006)), or by creating new visualization primitives and abstractions that incorporate uncertainty information, as suggested in (Pang, 2001).

When creating new visualization primitives to incorporate uncertainty information, the following methods are most popular: (a) free static variables such as colour, size, angle, texture, focus, fuzziness, transparency, edge crispness and blurring. However, this may add noise to the display; (b) free dynamic variables such as speed, blinking, duration and oscillation; (c) integrating geometry such as glyphs, labels and error bars. These additions need to be distinguishable from the base data; (d) utilising other human senses such as acoustics (Lodha et al., 1996).

## 5 UNCERTAINTY VISUALIZATION AND VISTA

The visualization research being carried out as part of VISTA aims to transform utility data from various sources into meaningful representations of the buried assets in order to assist better decision making. The complexity of integrating and depicting multiple underground assets is demonstrated in figure 1 which shows a map used by on-site teams to locate assets. The maps we create will improve current practices by highlighting the best methods to represent buried assets that also reflect the level of accuracy in the data.

<figure><img src="image_2.png" /><figcaption>Figure 1: An on-site map showing multiple utility assets.</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>Figure 2: The visualization framework.</figcaption></figure>

There are various sources of uncertainty in utility data, due to various factors; historical, organizational (both at the individual processes and at different levels of the organization) and technological. Legacy data, changes in organizational structures and new data management and surveying technologies all contribute to the uncertainty of underground assets.

We have established through a visualization questionnaire <fnref n="2" /> that there are a variety of user groups who are interested in visualizing the integrated utility data set, each with a different set of requirements which includes accuracy requirements. Therefore, it is not practical to aim for a one-map-for-all.

Our visualization framework (see figure 2) is based on initial feedback from the visualization questionnaire, whereby we have identified a set of generalized needs such as currency of data being presented, easy access to maps (e.g. via a web interface), layering of data and depiction of uncertainty information.

We have developed a prototype system (see figure 3) where we connect to a data source through a web service, then extract relevant details (e.g. attributes of the global schema that are of interest to the user). This information is then passed to the mapping stage where there are two different sets of rules; asset rules to transform data attributes from the global schema to geometry and other rules to map uncertainty data. The output is a 2D map displayed on the web. The user interacts with the display and the web server pulls the data and responds with a new map.

Our prototype system displays data using layers; a method often used in GIS and with which utility data users are familiar. Our initial work uses two methods to visualize the positional accuracy of assets: the blurring effect and colour bands.

Blurring (see figure 4) provides users with qualitative information about the spatial accuracy of underground assets. The more blurred a polyline, the less accurate its position. However, we are aware of the perceptual issues related to, for example, the number of levels of blurring that the user is able to distinguish. In addition, blurring in 2D with the presence of occlusion can be problematic. Over-plotting may result in certain areas looking less blurred or more certain than they are in reality.

Our second scheme is the traffic lights visualization. It uses a three-colour unified scheme to paint colour bands around all utility pipes indicating the confidence in the location of the assets. Figure 5 shows overall positional accuracy for a 5x5 km<fnref n="2" /> area containing three types of assets; gas, water and sewer where green

denotes certain, yellow denotes probable and red denotes uncertain (also see figure 6 which shows sewer pipes only, drawn in grey colour and the Ordnance Survey backdrop drawn in pink). We note that mapping different categorizations of locational confidence from different data sources can introduce inaccuracies; therefore uncertainty about uncertainty.

<figure><img src="image_4.png" /><figcaption>Figure 3: The prototype: a layer of water polylines is displayed over the Ordnance Survey backdrop.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 4: Blurring: different levels of positional accuracy are denoted by different levels of blurring.</figcaption></figure>

To ease the problem of occlusion in a 2D environment, we are working on a coordinated multiple views version of the prototype where each data set is visualized in a separate view or window. Currently, there is no national drawing standard for utility data maps (although there is the NWC/DoE standing tech-

<footnote n="2">²An online visualization questionnaire was designed [www.comp.leeds.ac.uk/mtu/Q.htm] to identify the visualization requirements of users. To date, we have received 43 responses from various types of organisations expressing their visualization and accuracy requirements.</footnote>
nical committee report no.25 for sewers and water mains records (STC25, 1980)). Thus, different organizations use their own internal drawing standards. In the case of a multi-view visualization system, each view uses the corresponding organizations drawing standards. However, for a coordinated exploration of underground assets, interactions in one view, e.g. a box selection in the gas view, is linked to interactions in other views (e.g. sewer and water).

<figure><img src="image_6.png" /><figcaption>Figure 5: The traffic lights scheme: uses a unified 3-colour scheme to denote uncertainty (pipelines have been removed for clarity).</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 6: Colour bands showing positional confidence for sewer pipes. Note that not all assets on the map have uncertainty information associated with them.</figcaption></figure>

Adding the third dimension to the display could overcome the occlusion problem in certain areas. Our questionnaire responses indicated that 77% of the respondents regard 3D mapping as an essential or useful feature of the visualization system. However, when asked about their preference on how to represent depth information, the same users preferred text labels to both 3D views

and 2D slices. Part of our future work is to assess the usefulness of 3D displays for utility maps. A recent workshop organized by a sister project <fnref n="3" /> indicated that a 3D view may be seen as more realistic and therefore more accurate than a 2D view, which could be misleading especially for depth information.

Visualization is itself a source of distortion, where accuracy of depiction is sometimes traded for clarity, e.g. placing drain pipes at different locations on a map to avoid occlusion. There is a long history of visual distortion in depiction, both gross distortion preserving only topological information, for example the London underground maps (Garland, 1994), or geometric distortion to reveal detail, such as the fisheye technique (Furnas, 1986). Arguably, these are not cases of 'uncertainty' - the data are known but the depiction process does not entirely tell the truth (perhaps a white lie?). Nonetheless, it may still be important to convey just where such distortions have been applied.

We also need to address the problem of visual interference, especially in congested maps, by combining visual marks. In addition, so far we have concentrated on the general needs of a wide range of users, focusing on positional accuracy. Our survey responses indicated that in 93% of cases, positional accuracy is regarded as essential or useful. Our future work will include other sources of uncertainty which could be qualitative (the survey indicated that the depiction of qualitative aspects of uncertainty such as attribute confidence, ambiguity, provenance, subjectivity and known omission is useful). Finally, we would like to further investigate the area of uncertainty and its link with judgement. Some work has been carried out on judgement under uncertainty (as described in the next section).

# 6 ANALYSIS AND EVALUATION OF UNCERTAINTY VISUALIZATION

Recently, there has been different studies to analyze uncertainty visualization. For example, there are empirical evaluations that apply Human Computer Interaction (HCI) methodologies in order to assess the effectiveness of uncertainty visualization and heuristic evaluations, e.g. (Zuk and Carpendale, 2006), that base their evaluation on perceptual and cognitive theories. Zuk & Carpendale apply perceptual theories which are well known in the visualization domain (e.g. work by Bertin, Tufte and Ware) in their analysis of eight uncertainty visualizations. They produced a simplified set of heuristics to analyze uncertainty visualization.

In general, evaluations of uncertainty visualizations assess whether the inclusion of uncertainty information in the representation is useful, i.e. if we adapt visualization to depict uncertainty, will users incorporate this information into decision making, or do they, as an early study (Tversky and Kahneman, 1974) has shown, rely on heuristics in making judgements. Other questions that are being addressed are concerned with whether different users incorporate uncertainty information in the same way, or does their level of expertise and prior knowledge play a role in how this information is incorporated in decision making.

Furthermore, such evaluations assess whether or note the inclusion of uncertainty information in representation improves decision making. They also study the degree of influence on users confidence and whether this degree of influence depends on the nature of the decision task or other factors. For example, an empirical evaluation (Deitrick and Edsall, 2006) indicated that this influence depends on how uncertainty is expressed (e.g. using texture or value).

<footnote n="3">³Mapping the Underworld: http://www mappingtheunderworld.ac.uk/</footnote>
The choice between an *implicit* approach where decisions are pre-mediated as opposed to an *explicit* approach where decisions are synthesized by the user interacting with the representation (e.g. map) is important. The implicit approach can ensure a unified decision made across a single user group, domain or user task. However, this method results in specialized representations which are only fit for specific purposes. Thus, the outcome of this type of representation cannot be extended to other decision making processes. In addition, implicit representations of uncertainty assume the visualization is the end product and that all sources of uncertainty have been captured. What about the subjective influence component of uncertainty (see section 2) and temporal issues, e.g. the time lag between the visualization product and the decision making process? From our own experience in the VISTA project, on-site teams use maps for guidance purposes and rely on other clues and information not provided on maps (e.g. surface furniture). When taking the explicit approach, users can distinguish between the raw data (arguably certain information) and uncertainty information (for a more accurate interpretation of the map), rather than a single pre-interpreted representation of the data. This means that the output of the visualization can serve many users having different purposes and each possibly making a different decision.

The study of human behaviour when reasoning with uncertainty is valuable to the analysis and design of uncertainty visualization. Therefore, efforts from the empirical domain of the visualization field need to be complemented by findings from perceptual and cognitive theories for better analysis of uncertainty visualization.

## ACKNOWLEDGEMENTS

The authors would like to thank the VISTA team in Leeds for their feedback. The School of Computing is a member of the VISTA project. Visualising integrated information on buried assets to reduce streetworks (VISTA) commenced in early 2006 and is a four year joint project with a consortium of academic and industry partners. UK water industry research (UKWIR) is the lead co-ordinating partner with Leeds and Nottingham Universities providing the research input. In addition there are over 20 utility and other industrial organisations. Thus, we would like to thank them for their input, in particular, Mike Shepherd from Thames Water and Simon Mears from Leica Geosystems for their useful comments on this paper. VISTA is principally funded though the Department of Trade and Industry Technology Programme with in kind contributions from the project partners. VISTA also builds on a pre-existing Engineering and Physical Sciences Research Council funded project, Mapping the Underworld (Grant EP/C014707/1).

## REFERENCES

*   Buttenfield, B., 1991. Visualizing cartographic metadata. Technical Report 91-26, NGCIA.

*   Buttenfield, B., 1994. Representing data quality. Cartographica 30(2&3), pp. 1-7. Special content, Mapping Data Quality.

*   Collins, C. and Penn, G., 2006. Leveraging uncertainty visualization to enhance multilingual chat. In: Proceedings of ACM Conference on Computer Supported Cooperative Work, pp. 61-62.

*   Deitrick, S. and Edsall, R., 2006. Progres in Spatial Data Handling. The Influence of Uncertainty Visualization on Decision Making: An Emperical Evaluation. SpringerLink.

*   Furnas, G., 1986. Generalized fisheye views. In Proceedings of the ACM SIG-CHI 86 Conference on Human Factors in Computing Systems pp. 16-23.

*   Garland, K., 1994. Mr Beck's Underground Map. Capital Transport Publishing.

*   Grigoryan, G. and Rheingans, P., 2002. Probabilistic surfaces: Point based primitives to show surface uncertainty. In: Proceedings of IEEE Visualization, IEEE CS Press, pp. 147-153.

*   Hunter, G. and Goodchild, M., 1993. Managing uncertainty in spatial databases: Putting theory into practice. Urban and Regional Information Systems Association URISA 5(2), pp. 55-62.

*   Johnson, C., 2004. Top scientific visualization research problems. IEEE Computer Graphics and Visualization: Visualization Viewpoints 24(4), pp. 13-17.

*   Johnson, C. and Sanderson, A., 2003. A next step: Visualizing errors and uncertainty. Computer graphics and applications IEEE 23(5), pp. 6-10.

*   Lodha, S., Wilson, C. and Sheehan, R., 1996. Listen: Sounding uncertainty visualization. In: Proceedings of IEEE Visualization, pp. 189-195.

*   MacEachren, A., 1992. Visualizing uncertain information. Cartographic Perspective 13(Fall), pp. 10-19.

*   MacEachren, A., Robinson, A., Hopper, S., Gardner, S., Murray, R., Gahegan, M. and Hetzler, E., 2005. Visualizing geospatial information uncertainty: What we know and what we need to know. Cartography and Geographic Information Science 32(3), pp. 139-160.

*   Olston, C. and Machinlay, J., 2002. Visualizing data with bounded uncertainty. Technical report, Standord University.

*   Pang, A., 2001. Visualizing uncertainty in geo-spatial data. In: In proceedings of the workshop on the intersections between geospatial information and information technology.

*   Plewe, B., 2002. The nature of uncertainty in historical geographical information. Transactions in GIS 6(4), pp. 431-456.

*   STC25, 1980. Sewers and water mains records. NWC/DoE standing technical committee report no. 25. WRC swindon. department of the environment and national water council.

*   STD, 2007. STDS spatial data transfer standards. http://mcmcweb.er.usgs.gov/sdts.

*   Stephenson, P., Wray, K., Kohlhammer, J. and Encarnao, L., 2006. Information quality visualization. Computer Graphik. INI-GraphicsNet.

*   Thomson, J., Hetzler, B., MacEachren, A., Gahegan, M. and Pavel, M., 2005. A typology for visualizing uncertainty. visualization and data analysis. In: in Proceedings of the SPIE, Visualization and Data Analysis, SPIE, IS&T, pp. 146-157.

*   Tory, M. and Moller, T., 2004. Rethinking visualization: a high-level taxonomy. In: Information Visualization, IEEE, pp. 151-158.

*   Tversky, A. and Kahneman, D., 1974. Judgement under uncertainty: heuristics and biases. Science 185(4157), pp. 1124-1131.

*   Zuk, T. and Carpendale, S., 2006. Theoretical analysis of uncertainty visualizations. In: Visualization and Data Analysis.