September 2015

NASA/TM–2015-218805

Multidimensional Risk Analysis: MRISK

Raymond McCollum

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia

Douglas Brown

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia

Sarah Beth O’Shea

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia

 

William Reith

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia

Jennifer Rabulan

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia

Graeme Melrose

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia
NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA scientific and technical information (STI) 

program plays a key part in helping NASA maintain 

this important role.

The NASA STI program operates under the auspices 

of the Agency Chief Information Officer. It collects, 

organizes, provides for archiving, and disseminates 

NASA’s STI. The NASA STI program provides access 

to the NTRS Registered and its public interface, the 

NASA Technical Reports Server, thus providing one 

of the largest collections of aeronautical and space 

science STI in the world. Results are published in both 

non-NASA channels and by NASA in the NASA STI 

Report Series, which includes the following report 

types:

 TECHNICAL PUBLICATION. Reports of 

completed research or a major significant phase of 

research that present the results of NASA 

Programs and include extensive data or theoretical 

analysis. Includes compilations of significant 

scientific and technical data and information 

deemed to be of continuing reference value. 

NASA counter-part of peer-reviewed formal 

professional papers but has less stringent 

limitations on manuscript length and extent of 

graphic presentations.

 TECHNICAL MEMORANDUM. 

Scientific and technical findings that are 

preliminary or of specialized interest, 

e.g., quick release reports, working 

papers, and bibliographies that contain minimal 

annotation. Does not contain extensive analysis.

 CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees.

 CONFERENCE PUBLICATION. 

Collected papers from scientific and technical 

conferences, symposia, seminars, or other 

meetings sponsored or 

co-sponsored by NASA.

 SPECIAL PUBLICATION. Scientific, 

technical, or historical information from NASA 

programs, projects, and missions, often 

concerned with subjects having substantial 

public interest.

 TECHNICAL TRANSLATION. 

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission.

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and feeds, 

providing information desk and personal search 

support, and enabling data exchange services.

For more information about the NASA STI program, 

see the following:

 Access the NASA STI program home page at 

http://www.sti.nasa.gov

 E-mail your question to help@sti.nasa.gov

 Phone the NASA STI Information Desk at 

757-864-9658

 Write to:

NASA STI Information Desk

Mail Stop 148

NASA Langley Research Center

Hampton, VA 23681-2199
National Aeronautics and 

Space Administration

Langley Research Center 

Hampton, Virginia 23681-2199

September 2015

NASA/TM–2015-218805

Multidimensional Risk Analysis: MRISK

Raymond McCollum

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia

Douglas Brown

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia

Sarah Beth O’Shea

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia

 

William Reith

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia

Jennifer Rabulan

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia

Graeme Melrose

Booz Allen Hamilton, Langley Research Center, Hampton, Virginia
Available from:

NASA STI Program / Mail Stop 148

NASA Langley Research Center

Hampton, VA 23681-2199

Fax: 757-864-6500

Acknowledgments 

Thank you to the ERA Project Manager, Dr. Fayette Collier, and the Deputy Project 

Manager, Ms. Gaudy Bezos-O’Connor for creating a work atmosphere that supports 

innovative project management ideas.

The use of trademarks or names of manufacturers in this report is for accurate reporting and does not constitute an 

official endorsement, either expressed or implied, of such products or manufacturers by the National Aeronautics and 

Space Administration.
4

Introduction

Informed decision making rests on the assessment of potential risks which allows projects 

and programs to focus their efforts on effective risk management (Kaplan and Garrick 1981). 

Accurate quantification of risks facilitates systematic and rigorous practice of risk management, 

which is particularly useful in project management. A relatively quantitative and generally 

accepted definition of risk is probability times consequence (Kaplan and Garrick, 1981; Rosa 

1998). Using this definition, quantifying risk is straightforward but flawed. Consequence is 

oftentimes a multidimensional proposition. The overall consequence may include a schedule 

dimension, safety dimension, financial dimension, etc. When faced with this issue, organizations 

typically average (weighted or otherwise) the consequence scores (Zayed et al. 2007); use 

Euclidean distance to combine the scores (Garvey 2008); or choose the maximum score 

(Oracle, 1999; Curtis and Carey, 2012). 

This paper focuses on an alternative multidimensional method using Statistical science to 

combine multiple dimensions together into one risk metric, called MRISK. Multidimensional Risk 

(MRISK) calculates the combined multidimensional score using Mahalanobis distance. MRISK 

quantifies and accounts for the least to most hazardous risks and, thus, better prioritizes risks. 

Likewise, MRISK accounts for covariance between consequence dimensions, since each 

consequence dimension may impact the other dimension. This helps differentiate when and why 

one should focus on certain risks. Accounting for covariance de-conflicts the interdependencies 

of consequence dimensions, providing a clearer depiction of risks. Additionally, in the event the 

dimensions are not correlated, Mahalanobis distance reduces to Euclidean distance normalized 

by the variance (Wilson and Martinez, 1997) and, therefore, represents the most flexible and 

optimal method to combine dimensions.

MRISK is currently being used in National Aeronautics and Space Administration’s (NASA’s) 

Environmentally Responsible Aviation (ERA) project to assess risk and prioritize scarce 

resources. The ERA case study is presented alongside specific examples with an explanation of 

MRISK, to demonstrate its effectiveness.

Multidimensional Risk

Industry accepted risk management practices use a Likert scale to translate qualitative 

assessments into a quantitative scale for consequence impact measure. (Hillson and Simon, 

2012). This paper will utilize Likert-scale data to illustrate MRISK although MRISK is applicable 

to all forms of quantitative data. In the given examples, three risk factors will be scored using a 

Likert scale of one to five, with one being associated with the least hazardous consequence and 

five being associated with the most hazardous consequence. It should be noted that scoring 

the risk factors appropriately is a vital step in a proper risk assessment. If the risk factor scores 

are inaccurate, MRISK will be no better than any other method However, defining a risk score 

assessment process is beyond the scope of this paper.

As previously mentioned, some alternatives to MRISK include choosing the maximum score 

(which will be referred to as the Maximization method), Euclidean method, and Averaging

method. 

The Maximization method assigns the highest consequence score amongst all the possible 

consequence dimensions to the consequence. For example, if three factors are rated as (1,5,1) 

the rating for the consequence is 5, and it is then combined with the probability score to 

determine the final risk rating. This implies that the consequence has a relatively high severity. 

The Maximization method may incorrectly take into account the weights of the other factors 

associated with the overall risk. More specifically, the Maximization method assumes all the risk 
5

components are highly correlated. By definition, using the highest consequence dimension 

score requires the assumption that the other consequence dimensions are equally high. This is 

an assumption of absolute correlation for all dimensions. For example, the risk rating for (1,5,1) 

is the same as (5,5,5). If the three consequences are perfectly correlated, then theoretically only 

(1,1,1), (2,2,2), (3,3,3), (4,4,4), and (5,5,5) would ever be observed. Likewise, highly correlated 

factors would have most scores with only marginal differences between them, such as (4,3,4) or 

(5,5,3). Larger differences such as (1,5,1) suggest the possibility of a weak correlation between 

the three factors. Risk management as a practice seeks to highlight risks. Therefore, the 

Maximization method may be excessively conservative. However, as the number of risk factors 

increase towards infinity, the logic of excessive conservatism falls apart, as seen in Table 1. 

(1,4) An overall consequence of 4 might be deemed 

acceptable for lower numbers of risk 

dimensions

(1,4,1)

(1,4,1,1)

(1,4,1,1,1)

... As number of dimensions rise absolute 

correlation as in the case of Maximization 

method has reduced merit

(1,4,1,1,......,1,1,1,1,1) 

Table 1: The Maximization method moves from conservatism to a biased estimate

The Averaging method assigns the mean consequence score among all the consequence 

dimensions to the consequence. For example, if three factors are rated as (1,5,1) the 

consequence is rated as a 2, suggesting relatively low severity. The weighted average of 

is given as,

If Wi=1/n for all i, then the formula reduces to the non-weighted average from Elementary 

Statistics (Triola, 2012). The Averaging method is a common practice among risk managers 

(Zayed et al., 2007). The average is a univariate parameter, and thus does not account for 

covariance of dimensions. Unlike the maximization method which has disproportional 

tendencies towards a single extreme, Averaging accounts for high and low extremes. For 

example, schedule increases generally go along with cost increases (PMBOK, 2000), and 

failure to capture covariance leaves important information out of risk analysis. Therefore, without 

taking covariance into consideration, risk managers may not have an accurate outlook on the 

risks at hand. Additionally, the Averaging method has disproportional tendencies towards the 

middle. Due to these tendencies, some risks could be deflated and considered a low severity 

risk. On the other hand, some risks could be inflated and considered a high severity risk. Both 

possibilities could be detrimental to a project.

Dimension Scores Average Many seemingly different 

combinations result in strikingly 

similar scores when using the 

Averaging method

(1,1,5) 2.33333

(3,3,2) 2.66667

(2,1,5) 2.66667

(1,4,3) 2.66667

(3,3,1) 2.33333

Table 2. The Average method has disproportional tendencies towards the center of the scale
6

The Euclidean method uses Euclidean distance to combine the consequence dimensions

(Garvey, 2008). The Euclidean method assigns co-planar distance amongst all the 

consequence dimensions to the consequence. For example, if three factors are rated at (1,5,1), 

the consequence would be rated as 3, suggesting a medium severity. The Euclidean distance 

for two vectors, and , each of length n is given as,

For this paper, the Euclidean method uses a ratio of the relative distances of the dimensions 

from the minimum, xmin, and maximum, xmax of the dimensions. The Euclidean method value, E, 

is found as follows.

Where, the dimensions are scored on a scale of [b,c] such that a=b-c. 

The Euclidean method is meant for multidimensional situations; therefore it is better suited

than the Averaging method for multidimensional risk analysis. Also, the Euclidean method does 

not have disproportional tendencies towards a single extreme as in the case of the Maximization 

method. However, the Euclidean method assumes the dimensions occupy the same plane, 

which may not be accurate.

This paper focuses on conducting Risk Analysis in the presence of multiple dimensions. 

With only one or two dimensions the conservative propensities present in the Maximization 

method may not be a problem. Likewise, the Averaging method’s disproportional tendency to 

the middle and ignorance of covariance could be acceptable with a low number of dimensions. 

Furthermore, assuming the dimensions are co-planar, as in the Euclidean method, might not 

make a difference with one or two dimensions. In the 1990s, risk was strictly deemed a two 

dimensional problem, namely probability and consequence, where consequence was one 

dimension as opposed to multidimensional (Rosa 1998, Renn 1998). However, risk 

management has come to recognize over the last two decades that multiple dimensions are a 

part of risk. In particular, NASA has expanded consequence to Cost, Schedule, and Technical 

(NASA/SP-2011-3421). Other areas like Finance often use five factors and the Department of 

Homeland Security (DHS) National Infrastructure Protection Plan (NIPP) and Sector Specific 

Plans have moved to three risk factors and are starting to consider a fourth Resilience risk 

factor (DHS, 2013).

The MRISK Procedure

MRISK is a procedure which uses multivariate metric, Mahalanobis distance in particular, to 

measure impact across several dimensions. In his 1936 paper, On the Generalized Distance in 

Statistics, P.C. Mahalanobis introduced this innovative method to measure distance between 

two vectors. This method takes into consideration the correlation of the vectors. Mahalanobis

distance essentially measures how many standard deviations away from a distribution a given 

point is; Mahalanobis also generalized the concept to multiple dimensions. MRISK is founded 

on Mahalanobis distance. The three primary advantages of using this generalized distance are:
7

 accounting for correlation between variables

 reverting to normalized Euclidean distance when correlation does not exist or when 

vectors occupy the same plane

 the ability to scale to infinite dimensions. In other words, the procedure will never lose 

validity as the dimensions grow.

Mahalanobis distance for two random vectors, and , of the same distribution with

covariance matrix, S, is defined as

S is a multidimensional matrix where each row is a consequence dimension and each 

column is also a consequence dimension. Take note that Mahalanobis distance simplifies to 

Euclidean distance when the covariance matrix is the identity matrix. Subsequently, for any 

scale, [b,c], where a is the range of the consequence scale and b is the minimum of the 

consequence scale so that a=c-b, the MRISK formula is defined as follows.

where,

So, is the Mahalanobis distance between the vector of consequence scores, x, and the 

vector of minimum possible consequence scores, , and is the Mahalanobis distances

between the vector of the consequence scores, , and the vector of maximum possible

consequence scores, . When using a Likert scale of one to five as previously mentioned 

and three consequence factors, and . The number of factors will 

change based on an organization’s preferences, but the range of one to five is representative of 

risk management as evidence by ERA. MRISK is not depend on any one scale. Therefore, 

always has a range of [0,1]; so in this case (1,1,1) maps to 0 and (5,5,5) to 1. Take 

note, that dmin is the numerator so that values near the top of the scale (e.g., 

(5,5,5)) reflect the highest score. 

NASA’s Environmentally Responsible Aviation, A Case Study

The Environmentally Responsible Aviation (ERA) project has provided an effective case 

study on the execution of MRISK in actual, risk-management practice. ERA was created in 2008 

and is part of NASA’s Aeronautics Research Mission Directorate’s (ARMD’s) Integrated 

Systems Research Program (ISRP). The ERA project was created to explore and document the 

feasibility, benefits, and technical risk of vehicle concepts and enabling technologies. The 

project invests in technologies with the potential to neutralize or reduce negative environmental 

impacts. The goal is to select vehicle concepts and technologies that improve fuel efficiencies, 

reduce harmful emissions, and lower noise levels of aircrafts. The ERA project uses several 

Integrated Technology Demonstrations (ITDs) to advance aircraft concepts and technologies 

that will reduce the impact of aviation on the environment. These demonstrations will focus on 

aircraft drag, weight, fuel usage, noise, and emission reduction. Engineers assessed 

environmentally friendly aircraft technologies and then matured the most promising ones to the 

point that they can be tested in a real world environment. 
8

For an ITD to test in a real world environment, several risks have to be reviewed such as 

design challenges, fabrication challenges, integration challenges, testing challenges, availability 

of personnel and facilities, etc. To determine how severe the risk is on the ITD, each risk is 

given a risk score. Each risk includes Cost, Schedule, and Technical dimensions scored in 

accordance with a Likert scale [1,5]. Risks are ultimately binned into green, yellow, and red 

categories as shown in Table 2 based on the combination of their multidimensional 

consequence score with the likelihood of the risk (one to five Likert scale). For the ultimate risk 

score green corresponds to a low impact risk, yellow a medium impact risk, and red a high 

impact risk.

Note that the risk scores will fluctuate over the course of the project. Many factors are time 

dependent. As certain phases of the project end risks associated with that phase end as well. 

Furthermore, new risks arise throughout the project, creating a very fluid and changing risk 

environment. For this paper a snapshot of the project timeline was taken and MRISK 

calculations were performed for that particular timestamp. The data used for those calculations 

are included in the appendix of this paper.

Risk, in the context of mission execution for the NASA ERA project, is operationally defined 

as (1) The scenario(s) leading to degraded performance with respect to one or more 

performance measures (e.g., scenarios leading to destruction of key assets; scenarios leading 

to exceeding the mass limits; scenarios leading to cost overruns; scenarios leading to schedule 

slippage). (2) The likelihood(s) (qualitative or quantitative) of those scenarios. (3) The 

consequence(s) (qualitative or quantitative) that would result if those scenarios were to occur 

(NASA, 2008).

ERA uses NASA’s Continuous Risk Management (CRM) process to manage risks 

continuously. The CRM process, as described in the Agency Risk Management Procedural 

Requirements, defines consequence as the worst credible potential result(s) of a risk (NASA, 

2008). Consequence scores range from one to five and ERA confines the risks to three major 

categories; Technical, Cost, and Schedule. Each of the consequence categories should be 

individually scored when making a determination of risk consequence; that is, each risk should 

have a Technical consequence, a Cost consequence, and a Schedule consequence, etc. 

Naturally, risk managers may use the Maximization method based on the assumption that 

the method ensures the risk is not underestimated. This approach does create the potential to 

overestimate as argued by its advocates. However, overestimation may limit the application of 

mitigation in risk management. Mitigation refers to the active de-escalation of risks (NASA, 

2008). The limiting factor for mitigation is usually budget, i.e. a finite-mitigation budget forces the 

risk manager to limit the scope or number of risks mitigated by the project. Therefore, estimation 

may de-prioritize dangerous risks and over-prioritize low risks. The Averaging method, has the 

disproportional tendency to rate risks at a medium severity. If the majority of risks are in the 

middle of the risk scale, risk managers may have a difficult time distinguishing the criticality of 

risks. The Euclidean method assumes co-planar dimensions, which may not be accurate, and 

thus could rate risks incorrectly. Also, this approach does not take into account the weights of 

the other risk factors and the correlation between Technical, Cost, and Schedule risks. For 

example, higher costs tend to follow schedule increases, schedule decreases may create more 

technical risk, tighter schedules tend to have greater cost risk, and difficult technical challenges 

tend to take longer and cost more to execute. Without the correlation between the different 

consequence categories, a less severe risk may be prioritized above a risk of higher severity, 

resulting in unnecessary allocations of resources for risk mitigations strategies. For example, 

the standard approach would imply a risk with Technical, Cost, and Schedule consequences of 

(4,4,4) would be considered an equal priority to a risk with consequences of (1,1,4).

In addition to the intuitive notion that consequences are correlated, several NASA sources 

acknowledge that the consequence components of Cost and Schedule in particular are 

interdependent and that risk assessments should account for covariance. For example, the 
9

2008 Goddard Space Flight Center (GSFC) Symposium presentation Perspectives on NASA 

Mission Cost and Schedule Performance Trends by David Bearden stated “While Significant 

Variability is Evident, for Every 10% of Schedule Growth, there is a Corresponding 12% 

Increase in Cost”. David Bearden presents physical evidence in his presentation that shows 

there is a correlation between schedule growth and increases in cost. 

Consequently, NASA leadership developed the NASA Schedule Management Handbook

which expressly states “NPR 7120.5 requires that the project’s schedule baseline be integrated 

with the budget and technical baselines to form an overall project integrated baseline. This 

correlation is essential to ensure that adequate resources are available to accomplish the work 

when it is scheduled. Without this correlation and validation the project IMS [Integrated Master 

Schedule] loses credibility” (NASA, January 2010).

Additionally, NASA developed the Joint Cost and Schedule Confidence Level (JCL) in order 

to encourage more effective project planning “earlier in the life cycle and to support 

development of more accurate estimates”. NPR 7120.5, NASA Space Flight Program and 

Project Management Handbook defines “Joint Cost and Schedule Confidence Level. (1) The 

probability that the program/project cost will be equal to or less than the targeted cost and that 

schedule will be equal to or less than the targeted schedule date. (2) A process and product that 

helps inform management of the likelihood of a project’s programmatic success. (3) A process 

that combines a project’s cost, schedule, and risk into a complete picture” (NASA, 2014).

MRISK was conceived for a multidimensional risk analysis and thereby follows the 

underlying intent of the JCL by combining Cost and Schedule dimensions as well as other 

dimensions into a single metric that properly accounts for correlation between dimensions. The 

procedure accounts for covariance amongst multiple risk dimensions Additionally, the 

application problems associated with Averaging, Maximization, and Euclidean analysis made 

the case for MRISK to the ERA project management team. That is to say, the ERA project 

converted from the Maximization method to MRISK after coming to understand the benefits of 

MRISK.

ERA Examples

In the following examples, the MRISK formula result is rounded and combined with the 

likelihood using Table 2 to determine the risk rating. Table 2 is not a necessary step for MRISK. 

MRISK has the ability to measure likelihood along with all the other consequence dimensions. 

However, Table 2 is an example of a customary figure in the practice of risk management 

(Conrow 2003) and this paper shows that MRISK is adaptable to this traditional practice used in 

ERA.

Furthermore, to determine the overall severity rating of a risk, likelihood score and the 

consequence score are taken into consideration. Thus far, we have discussed different methods 

of combining the consequence scores. In particular, we presented the Maximization method, 

averaging, Euclidean distance, and MRISK. Once the consequence and likelihood scores are 

selected, Table 2 is used to determine the overall risk rating. For example, with a consequence 

score of 3 and a likelihood score of 2, the risk would be given a score of 18, which is of medium 

severity.
10

Risk Consequence Likelihood

1 2 3 4 5

5 10 16 20 23 25

4 7 13 18 22 24

3 4 9 15 19 21

2 2 6 11 14 17

1 1 3 5 8 12

Table 3. ERA Risk Scoring Matrix

Example

Risk Consequence Likelihood

1 2 3 4 5

5

4

3

2

1

Table 4. Mapping of (2,2) to 6

Example of MRISK Without Correlation

In the event that correlation does not exist, the variance-covariance matrix will be a matrix 

with only the diagonal elements, the variances. Subsequently,
11

So, this is a normalized Euclidean distance between and , In this case, the Euclidean 

distance is normalized with the variances. Furthermore, if all of the variances equal one, this will 

be Euclidean distance. This shows the flexibility of MRISK, which can account for both 

Mahalanobis and Euclidean distance.

Example

Risk Consequence Likelihood

1 2 3 4 5

5

4

3

2

1

Table 5. Mapping of (2,2) to 6
12

Notice that the result is the same as MRISK with covariance. The results are the same, 

because the ERA covariance matrix was used, and the ERA risks have relatively low 

covariance.

Dispersion Mapping of ERA Data

The ERA project implements continuous risk management, which means the number and 

types of risks change throughout the life of the project. As risks are mitigated, become obsolete, 

or as new risks are created the list of risks will change. Therefore, the covariance between Cost, 

Schedule, and Technical will change throughout the life of the project. The appendix contains 

the summary of 179 risks on the ERA project at the time of writing this paper. Several of these 

risks will be highlighted due to their particular importance for comparing MRISK to alternative 

methods, Maximization, Euclidean, and Averaging.

Figure 1. 3-D plot of risks rated with MRISK 

method

Figure 3. 3-D plot of risks rated with Euclidean 

method

Figure 2. 3-D plot of risks rated with 

Maximization method

Figure 4. 3-D plot of risks rated with Averaging 

method

Figure 1 through 4 show three dimensional plots with axes of Cost, Schedule, and Technical 

for each method of determining the combined consequence score. The size of the point reflects
13

the likelihood score with a score of one being the smallest point (and lowest likelihood) and five 

being the largest point (and highest likelihood). The colors demonstrate the overall risk rating, 

which combines the likelihood and the consequence score. Green symbolizes a low risk, yellow 

symbolizes a medium risk, and red symbolizes a high risk. From these plots, we can see that 

using the Maximization method results in a large proportion of medium and high risks. When 

dealing with risks, being more conservative may seem like the right answer; however, when 

mitigation resources are scarce, taking an overly conservative approach may exhaust resources 

before mitigation can buy down risks. The Euclidean distance values are calculated just as the 

MRISK values are, except the covariance matrix is the identity matrix. Additionally, Euclidean 

distance and MRISK are very similar. This is due to the weak correlation between the risk 

factors (i.e., MRISK behaves in a similar fashion to Euclidean distance for the ERA project). 

Figure 5. Histogram showing dispersion of risks 

rated with MRISK method

Figure 7. Histogram showing dispersion of risks 

rated with Euclidean method

Figure 6. Histogram showing dispersion of risks 

rated with Maximization method

Figure 8. Histogram showing dispersion of risks 

rated with Maximization method
14

In Figure 5 through 8, the dispersion of consequence scores is shown. The Maximization

method has disproportional tendencies towards the high extreme. The other three methods

disproportionally tend to fall in the middle, but MRISK has the most even dispersion among the 

methods. This brings us to the point, that MRISK has the least systematic tendencies. Because 

of this dispersion, using this method more clearly defines the cutoffs between green, yellow, and 

red. That is to say, MRISK allows one to better distinguish between a low risk and a medium 

risk or a medium and a high risk. Therefore, one can better allocate scarce resources towards 

right risks.

Next, let us consider an example of MRISK calculations and take a deeper look at the 

differences between the various methods which may not be readily apparent in the charts

above. 

Example: Risk ID 236

Here the Maximization method returns a risk score of 8 or low risk, but both MRISK and 

Euclidean distance return risk score of 12 or medium risk. This example is important to highlight, 

because it is a counter example to the argument that the Maximization method always returns a 

higher risk score and therefore, is more conservative approach in all situations.

Conclusions

An accurate portrayal of risks is a key step in informed decision-making. A key challenge of 

risk assessment in the modern era is multidimensionality. Prior to 1990, multidimensional risk 

was not an issue, but in the modern era it has become an issue that all risk managers must 

address. MRISK is an alternative multidimensional approach to dealing with risk management 

utilizing Mahalanobis distance that lacks the disproportional tendencies of legacy methods. 

Maximization, Averaging, and Euclidean methods are alternative, legacy methods used in risk 

management with deficiencies in the presence of multidimensionality. Specifically, the 

Maximization method does not properly account for covariance and has disproportional 

tendencies towards the extreme of the risk scale. Averaging is not a multidimensional metric

and has disproportional tendencies towards the center of the risk scale. Euclidean is a 

multidimensional metric, but does not account for covariance among risk dimensions.
15

In contrast, MRISK is a sound methodology that accounts for covariance among several 

dimensions and does not exhibit disproportional tendencies apparent in the other legacy 

methods. MRISK is scalable to any number of risk dimensions. MRISK is effective whether the

risk dimensions are highly correlated, weakly correlated, or have no correlation at all, because it 

calculates the covariance between the risk dimensions and uses the covariance in its 

calculations. As a result, MRISK is fully compliant with organizational mandates regarding 

covariance between dimensions. As cited in this paper, organizations such as NASA often 

require covariance integration within risk analysis, e.g. Covariance(Cost, Schedule). MRISK 

represents the best multidimensional method available, because of its reliance on Mahalanobis 

distance. MRISK accounts for covariance when present, via a covariance matrix. MRISK also 

accounts for lack of covariance by reducing to the Euclidean method when covariance is not 

present.

MRISK was successfully used as part of the risk management process on NASA’s ERA 

project. MRISK allowed ERA risk managers to more accurately allocate resources on the 

project. The MRISK method integrated seamlessly with traditional risk management practices, 

such as 5x5 view graphs and risk dispersion maps. MRISK has been an effective improvement 

on the risk management process for NASA and could be applied in any organization with similar 

challenges. 
16

Appendix: Risk Table

The following table shows the observed combinations of consequence scores and their 

counts. L = Likelihood, C = Cost, S = Schedule, and T = Technical under Risk Dim (Risk 

Dimensions).

Risk Dim 

(L,C,S,T)

Count Risk Dim 

(L,C,S,T)

Count Risk Dim 

(L,C,S,T)

Count

(1,1,1,1) 64 (2,1,2,2) 3 (3,1,3,3) 2

(1,1,1,2) 1 (2,1,2,3) 2 (3,1,4,1) 1

(1,1,1,3) 4 (2,1,3,1) 11 (3,1,4,2) 2

(1,1,1,5) 1 (2,1,3,2) 1 (3,1,4,4) 1

(1,1,2,1) 4 (2,1,4,3) 1 (3,1,5,1) 1

(1,1,2,2) 1 (2,1,5,1) 1 (3,1,5,3) 1

(1,1,2,3) 1 (2,1,5,5) 1 (3,2,1,3) 1

(1,1,2,5) 2 (2,2,1,1) 1 (3,2,1,4) 1

(1,1,3,1) 2 (2,2,1,5) 1 (3,2,3,1) 1

(1,1,3,2) 1 (2,2,2,1) 1 (3,2,3,3) 1

(1,1,3,3) 1 (2,2,2,2) 2 (3,2,3,4) 1

(1,1,4,1) 2 (2,2,3,3) 1 (3,2,4,4) 1

(1,1,4,3) 1 (2,2,5,3) 1 (3,2,5,4) 1

(1,2,1,3) 1 (2,3,2,3) 1 (3,2,5,5) 1

(1,3,3,5) 1 (2,4,1,1) 1 (3,3,2,1) 1

(1,3,5,4) 1 (2,5,3,4) 1 (3,3,3,2) 1

(1,4,1,4) 1 (3,1,1,1) 5 (3,3,3,3) 2

(1,4,4,4) 1 (3,1,1,4) 1 (3,3,3,4) 1

(1,5,1,4) 1 (3,1,2,1) 2 (3,4,4,5) 1

(1,5,3,3) 1 (3,1,2,2) 1 (3,4,5,5) 1

(2,1,1,1) 19 (3,1,2,3) 1 (4,1,1,1) 1

(2,1,1,3) 3 (3,1,2,4) 1 (4,1,3,1) 5

(2,1,2,1) 5 (3,1,3,1) 5 (4,1,4,1) 1

(4,2,1,3) 1 (4,2,3,4) 2 (4,1,4,3) 1

(4,2,1,4) 1 (4,2,5,4) 1 (4,1,5,1) 1

(4,2,2,4) 2 (4,4,3,4) 1 (5,2,3,1) 1

(4,2,3,2) 1 (5,1,2,4) 1
17

References

Bearden, David. Perspectives on NASA Mission Cost and Schedule Performance Trends. 

Presentation at GSFC Symposium, June 3, 2008.

Conrow, E. Effective Risk Management: Some Keys to Success, Second Edition. American 

Institute of Aeronautics and Astronautics Inc., 2003.

Curtis, P. and Carey, M. Risk Assessment in Practice, Committee of Sponsoring Organization of 

the Treadway Commission, October, 2012.

Garvey, P.R., 2008, Analytical Methods for Risk Management: A Systems Engineering 

Perspective, Chapman-Hall/CRC Press, Taylor & Francis Group (UK), Boca Raton, London, 

New York, ISBN: 1584886374.

A Guide to the Project Management Body of Knowledge (PMBOK guide), 2000 Edition. Project 

Management Institute, Inc.: Pennsylvania.

Hillson, D. and Simon, P. Practical Project Risk Management: The ATOM Methodology, Second 

Edition, Management Concepts Press, 2012

Johnson, D.E. Applied Multivariate Methods for Data Analytics, Duxbury Press, 1998.

Johnson, R.A. and Wichern, D.W. Applied Multivariate Statistics, 5th Edition. Prentice Hall, 

2001. 

Kaplan, S. and Garrick, B.J., On the Quantitative Definition of Risk, Risk Analysis, Vol. 1, No. 1, 

1981.

NASA, NASA Space Flight Program and Project Management Handbook, Washington, D.C. 

January 2014.

NASA, NASA/SP-2010-3403, Schedule Management Handbook, Washington, D.C., January 

2010.

NASA Office of Safety and Mission Assurance, Agency Risk Management Procedural 

Requirements (NPR 8000.4A), December 16, 2008.

NASA Office of Safety and Mission Assurance, NASA/SP-2010-576, NASA Risk-Informed 

Decision Making Handbook, April 2010.

Oracle USA, Inc. Primavera P6 Enterprise Project Portfolio Management, Redwood City, CA, 

1999.

Stamatelatos, M. and Dezfuli, H. NASA/SP-2011-3421, Probabilistic Risk Assessment 

Procedures Guide for NASA Managers and Practitioners, Second Edition, Washington D.C., 

December 2011.

Triola, M. Elementary Statistics, Twelfth Edition, Pearson: New Jersey, 2012.

Wilson, D.R. and Martinez, T.R. Improved Heterogeneous Distance Functions, Journal of 

Artificial Intelligence Research 6 (1997), 1-34.

Zayed, T., Amer, M., and Pan, J. Assessing risk and uncertainty inherent in Chinese highway 

projects using AHP, International Journal of Project Management, 26 (2008), 408-419.
REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

2. REPORT TYPE 

Technical Memorandum

 4. TITLE AND SUBTITLE

Multidimensional Risk Analysis: MRISK

5a. CONTRACT NUMBER

 6. AUTHOR(S)

McCollum, Raymond; Brown, Douglas; O'Shea, Sarah Beth; Reith, 

William; Rabulan, Jennifer; Melrose, Graeme

 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

NASA Langley Research Center

Hampton, VA 23681-2199

 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration

Washington, DC 20546-0001

 8. PERFORMING ORGANIZATION

 REPORT NUMBER

L-20514

10. SPONSOR/MONITOR'S ACRONYM(S)

NASA

13. SUPPLEMENTARY NOTES

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified - Unlimited

Subject Category 81

Availability: NASA STI Program (757) 864-9658

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email: help@sti.nasa.gov)

14. ABSTRACT

Multidimensional Risk (MRISK) calculates the combined multidimensional score using Mahalanobis distance. MRISK 

accounts for covariance between consequence dimensions, which de-conflicts the interdependencies of consequence 

dimensions, providing a clearer depiction of risks. Additionally, in the event the dimensions are not correlated, Mahalanobis 

distance reduces to Euclidean distance normalized by the variance and, therefore, represents the most flexible and optimal 

method to combine dimensions. MRISK is currently being used in NASA's Environmentally Responsible Aviation (ERA) 

project to assess risk and prioritize scarce resources. 

15. SUBJECT TERMS

Covariance; Mahalanobis Distance; Multidimensional Analysis; Project Management; Risk; Risk Assessment; Risk 

Management; Statistics

18. NUMBER

 OF 

 PAGES

19

19b. TELEPHONE NUMBER (Include area code)

(757) 864-9658

a. REPORT

U

c. THIS PAGE

U

b. ABSTRACT

U

17. LIMITATION OF 

 ABSTRACT

UU

Prescribed by ANSI Std. Z39.18

Standard Form 298 (Rev. 8-98)

3. DATES COVERED (From - To)

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

 699959.02.07.07.03

11. SPONSOR/MONITOR'S REPORT

 NUMBER(S)

NASA-TM-2015-218805

16. SECURITY CLASSIFICATION OF:

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 

gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 

collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person 

shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

01- 09 - 2015