Managing Information Security Risk: Organization, Mission, and Information System View
Appendix H); and (vii) the general organizational directions and guidance on appropriate responses to risk. In addition > 3.4 MONITORING RISK > TASK 4-1: Develop a risk monitoring strategy for the organization that includes the purpose, type, and (part 2)

For
example, if the desired outcome for organizations is to be less susceptible to advanced persistent threats, this may be
challenging to measure since these types of threats are, by definition, very difficult to detect. Even when organizations
are able to establish effectiveness criteria, it is often difficult to obtain criteria that are quantifiable. Therefore, it may
become a matter of subjective judgment as to whether the implemented risk response measures are ultimately effective. Moreover, even if quantifiable effectiveness criteria are provided, it may be difficult to determine if the information
provided satisfies the criteria. If organizations determine that risk response measures are not effective, then it may be
necessary to return to the risk response step. Generally, for effectiveness failures, organizations cannot simply return to
the implementation portion of the risk response step. Therefore, depending on the reason for the lack of effectiveness,
organizations revisit all portions of the risk response step (i.e., development, evaluation, decision, and implementation)
and potentially the risk assessment step. These activities may result in organizations developing and implementing
entirely new risk responses. Monitoring Changes
In addition to compliance monitoring and effectiveness monitoring, organizations monitor changes to organizational
information systems and the environments in which those systems operate. Monitoring changes to information systems
and environments of operation is not linked directly to previous risk response measures but it is nonetheless important
to detect changes that may affect the risk to organizational operations and assets, individuals, other organizations, and
the Nation. Generally, such monitoring detects changes in conditions that may undermine risk assumptions (articulated
in the risk framing step). •
Information System: Changes can occur in organizational information systems (including hardware, software, and
firmware) that can introduce new risk or change existing risk. For example, updates to operating system software
can eliminate security capabilities that existed in earlier versions, thus introducing new vulnerabilities into
organizational information systems. Another example is the discovery of new system vulnerabilities that fall
outside of the scope of the tools and processes available to address such vulnerabilities (e.g., vulnerabilities for
which there are no established mitigations). •
Environments of Operation: The environments in which information systems operate can also change in ways that
introduce new risk or change existing risk. Environmental and operational considerations include, but are not
limited to, missions/business functions, threats, vulnerabilities, mission/business processes, facilities, policies,
legislation, and technologies. For example, new legislation or regulations could be passed that impose additional
requirements on organizations. This change might affect the risk assumptions established by organizations. Another example is a change in the threat environment that reports new tactics, techniques, procedures, or
increases in the technical capabilities of adversaries. Organizations might experience reductions in available
resources (e.g., personnel or funding), which in turn results in changing priorities. Organizations might also
experience changes in the ownership of third-party suppliers which could affect supply chain risk. Mission
changes may require that organizations revisit underlying risk assumptions. For example, an organization whose
mission is to collect threat information on possible domestic terrorist attacks and share such information with
appropriate federal law enforcement and intelligence agencies may have its scope changed so that the organization
is responsible for also sharing some of the information with local first responders. Such a change could affect
assumptions regarding the security resources such users may have at their disposal. Changes in technology may
also affect the underlying risk assumptions established by organizations. Unlike other types of change, technology
changes may be totally independent of organizations, but still affect the risk organizations must address. For
example, improvements in computing power may undermine assumptions regarding what constitutes sufficiently
strong means of authentication (e.g., number of authentication factors) or cryptographic mechanism. Automated Versus Manual Monitoring
Broadly speaking, organizations can conduct monitoring either by automated or manual methods. Where automated
monitoring is feasible, it should be employed because it is generally faster, more efficient, and more cost-effective than
manual monitoring. Automated monitoring is also less prone to human error. However, not all monitoring can take
advantage of automation. Monitoring conducted at Tier 3 generally lends itself to automation where activities being
monitored are information technology-based. Such activities can usually be detected, tracked, and monitored through
the installation of appropriate software, hardware and/or firmware. To ensure that automated processes, procedures,
and/or mechanisms supporting monitoring activities are providing the information needed, such processes, procedures,
and mechanisms should be appropriately validated, updated and monitored. Compliance monitoring can be supported
by automation when the risk mitigation measures being validated are information technology-based (e.g., installation of
firewalls or testing of configuration settings on desktop computers). Such automated validation can often check
whether risk mitigation measures are installed and whether the installations are correct. Similarly, effectiveness
monitoring may also be supported by automation. If the threshold conditions for determining the effectiveness of risk
response measures are predetermined, then automation can support such effectiveness monitoring.