This paper gives the main definitions relating to dependability, a generic concept including as special case such attributes as reliability, availability, safety, confidentiality, integrity, maintainability, etc. Basic definitions are given first. They are then commented upon, and supplemented by additional definitions, which address the threats to dependability (faults, errors, failures), and the attributes of dependability. The discussion on the attributes encompasses the relationship of dependability with security, survivability and trustworthiness. Key words: Dependability, availability, reliability, safety, confidentiality, integrity, maintainability, security, survivability, trustworthiness, faults, errors, failures. The delivery of correct computing and communication services has been a concern of their providers and users since the earliest days. In the July 1834 issue of the Edinburgh Review, Dr. Dionysius Lardner published the article "Babbage's calculating engine", in which he wrote: "The most certain and effectual check upon errors which arise in the process of computation, is to cause the same computations to be made by separate and independent computers; and this check is rendered still more decisive if they make their computations by different methods". 2 It must be noted that the term "computer" in the previous quotation refers to a person who performs computations, and not the "calculating engine". The first generation of electronic computers (late 1940's to mid-50's) used rather unreliable components, therefore practical techniques were employed to improve their reliability, such as error control codes, duplexing with comparison, triplication with voting, diagnostics to locate failed components, etc. At the same time J. von Neumann [von Neumann 1956] , E. F. Moore and C. E. Shannon [Moore & Shannon 1956] , and their successors developed theories of using redundancy to build reliable logic structures from less reliable components, whose faults were masked by the presence of multiple redundant components. The theories of masking redundancy were unified by W. H. Pierce as the concept of failure tolerance in 1965 [Pierce 1965] . In 1967, A. Avižienis integrated masking with the practical techniques of error detection, fault diagnosis, and recovery into the concept of faulttolerant systems [Avižienis 1967 ]. In the reliability modeling field, the major event was the introduction of the coverage concept by Bouricius, Carter and Schneider [Bouricius et al. 1969] . Work on software fault tolerance was initiated by Elmendorf [Elmendorf 1972] , later it was complemented by recovery blocks [Randell 1975] , and by N-version programming [Avižienis & Chen, 1977] . [Laprie 1992], in which the English text was also translated into French, German, Italian, and Japanese. In this book, intentional faults (malicious logic, intrusions) were listed along with accidental faults (physical, design, or interaction faults). Exploratory research on the integration of fault tolerance and the defenses against deliberately malicious faults, i.e., security threats, was started in the mid-80's [Dobson & Randell 1986] , [Joseph & Avižienis 1988] , [Fray et al. 1986] . The first IFIP Working Conference on Dependable Computing for Critical Applications (DCCA) was held in 1989. This and the six Working Conferences that followed fostered the interaction of the dependability and security communities, and advanced the integration of security (confidentiality, integrity and availability) into the framework of dependable computing. Since 2000, the DCCA Working Conference together with the FTCS became parts of the International Conference on Dependable Systems and Networks (DSN).  Dependability and Its Threats: A Taxonomy In this section we present a basic set of definitions (in bold typeface) that will be used throughout the entire discussion of the taxonomy of dependable computing. The definitions are general enough to cover the entire range of computing and communication systems, from individual logic gates to networks of computers with human operators and users. A system in our taxonomy is an entity that interacts with other entities, i.e., other systems, including hardware, software, humans, and the physical world with its natural phenomena. These other systems are the environment of the given system. The system boundary is the common frontier between the system and its environment. Computing and communication systems are characterized by four fundamental properties: functionality, performance, dependability, and cost. Those four properties are collectively influenced by two other properties: usability and adaptability. The function of such a system is what the system is intended to do and is described by the functional specification in terms of functionality and performance. Dependability and cost have separate specifications. The behavior of a system is what the system does to implement its function and is described by a sequence of states. The total state of a given system is the set of the following states: computation, communication, stored information, interconnection, and physical condition. The structure of a system is what enables it to generate the behavior. From a structural viewpoint, a system is a set of components bound together in order to interact, where each component is another system, etc. The recursion stops when a component is considered to be atomic: any further internal structure cannot be discerned, or is not of interest and can be ignored. The service delivered by a system (the provider) is its behavior as it is perceived by its user(s); a user is another system that receives service from the provider. The part of the provider's system boundary where service delivery takes place is the service interface. The part of the provider's total state that is perceivable at the service interface is its external state; the remaining part is its internal state. The delivered service is a sequence of the provider's external states. We note that a system may sequentially or simultaneously be a provider and a user with respect to another system, i.e., deliver service to and receive service from that other system. It is usual to have a hierarchical view of a system structure. The relation is composed of, or is decomposed into, induces a hierarchy; however it relates only to the list of the system components. A hierarchy that takes into account the system behavior is the relation uses [Parnas 1974, Ghezzi et al. We have up to now used the singular for function and service. A system generally implements more than one function, and delivers more than one service. Function and service can be thus seen as composed of function items and of service items. For the sake of simplicity, we shall simply use the plural -functions, services -when it is necessary to distinguish several function or service items. Correct service is delivered when the service implements the system function. A service failure is an event that occurs when the delivered service deviates from correct service. A service fails either because it does not comply with the functional specification, or because this specification did not adequately describe the system function. A service failure is a transition from correct service to incorrect service, i.e., to not implementing the system function. The period of delivery of incorrect service is a service outage. The transition from incorrect service to correct service is a service restoration. The deviation from correct service may assume different forms that are called service failure modes and are ranked according to failure severities. A detailed taxonomy of failure modes is presented in Section 4. Since a service is a sequence of the system's external states, a service failure means that at least one (or more) external state of the system deviates from the correct service state. The deviation is called an error. The adjudged or hypothesized cause of an error is called a fault. In most cases a fault first causes an error in the service state of a component that is a part of the internal state of the system and the external state is not immediately affected. For this reason the definition of an error is: the part of the total state of the system that may lead to its subsequent service failure. It is important to note that many errors do not reach the system's external state and cause a failure. A fault is active when it causes an error, otherwise it is dormant. When the functional specification of a system includes a set of several functions, the failure of one or more of the services implementing the functions may leave the system in a degraded mode that still offers a subset of needed services to the user. The specification may identify several such modes, e.g., slow service, limited service, emergency service, etc. Here we say that the system has suffered a partial failure of its functionality or performance. Development failures and dependability failures that are discussed in Section 4 also can be partial failures. Dependability and Its Threats: A Taxonomy5 The general, qualitative, definition of dependability is: the ability to deliver service that can justifiably be trusted. This definition stresses the need for justification of trust. The alternate, quantitative, definition that provides the criterion for deciding if the service is dependable is: dependability of a system is the ability to avoid service failures that are more frequent and more severe than is acceptable to the user(s). As developed over the past three decades, dependability is an integrating concept that encompasses the following attributes: • availability: readiness for correct service; • reliability: continuity of correct service; • safety: absence of catastrophic consequences on the user(s) and the environment; • confidentiality: absence of unauthorized disclosure of information; • integrity: absence of improper system alterations; • maintainability: ability to undergo, modifications, and repairs. Security is the concurrent existence of a) availability for authorized users only, b) confidentiality, and c) integrity with 'improper' meaning 'unauthorized'. The dependability specification of a system must include the requirements for the dependability attributes in terms of the acceptable frequency and severity of failures for the specified classes of faults and a given use environment. One or more attributes may not be required at all for a given system. The taxonomy of the attributes of dependability is presented in Section 5. Over the course of the past fifty years many means to attain the attributes of dependability have been developed. Those means can be grouped into four major categories: • fault prevention: means to prevent the occurrence or introduction of faults; • fault tolerance: means to avoid service failures in the presence of faults; • fault removal: means to reduce the number and severity of faults; • fault forecasting: means to estimate the present number, the future incidence, and the likely consequences of faults. Fault prevention and fault tolerance aim to provide the ability to deliver a service that can be trusted, while fault removal and fault forecasting aim to reach confidence in that ability by justifying that the functional and dependability specifications are adequate and that the system is likely to meet them.  THE TAXONOMY OF FAULTS In this and the next section we present the taxonomy of threats that may affect a system during its entire life. The life cycle of a system consists of two phases: development and use. The development phase includes all activities from presentation of the user's initial concept to the decision that the system has passed all acceptance tests and is ready to be deployed for use in its user's environment. During the development phase the system is interacting with the development environment and development faults may be introduced into the system by the environment. The development environment of a system consists of the following elements: 1. the physical world with its natural phenomena; 2. human developers, some possibly lacking competence or having malicious objectives; 3. development tools: software and hardware used by the developers to assist them in the development process; 4. production and test facilities. The use phase of a system's life begins when the system is accepted for use and starts the delivery of its services to the users. Use consists of alternating periods of correct service delivery (to be called service delivery), Taxonomy   7 service outage, and service shutdown. A service outage is caused by a service failure. It is the period when incorrect service (including no service at all) is delivered at the service interface. A service shutdown is an intentional halt of service by an authorized entity. Maintenance actions may take place during all three periods of the use phase. During the use phase the system interacts with its use environment and may be adversely affected by faults originating in it. The use environment consists of the following elements: 1. the physical world with its natural phenomena; 2. the administrators (including maintainers): entities (humans, other systems) that have the authority to manage, modify, repair and use the system; some authorized humans may lack competence or have malicious objectives; 3. the users: entities that receive service at the service interfaces; 4. the providers: entities that deliver services to the system at its service interfaces; 5. the fixed resources: entities that are not users, but provide specialized services to the system, such as information sources (e.g., GPS, time, etc.), communication links, power sources, cooling airflow, etc. 6. the intruders: malicious entities that have no authority but attempt to intrude into the system and to alter service or halt it, alter the system's functionality or performance, or to access confidential information. They are hackers, malicious insiders, agents of hostile governments or organizations, and info-terrorists. As used here, the term maintenance, following common usage, includes not only repairs, but also all modifications of the system that take place during the use phase of system life. Therefore maintenance is a development process, and the preceding discussion of development applies to maintenance as well. The various forms of maintenance are summarized in  It is noteworthy that repair and fault tolerance are related concepts; the distinction between fault tolerance and maintenance in this paper is that maintenance involves the participation of an external agent, e.g., a repairman, test equipment, remote reloading of software. Furthermore, repair is part of fault removal (during the use phase), and fault forecasting usually considers repair situations. All faults that may affect a system during its life are classified according to eight basic viewpoints that are shown in Figure 3. The classification criteria are as follows: 1. The phase of system life during which the faults originate: • development faults that occur during (a) system development, (b) maintenance during the use phase, and (c) generation of procedures to operate or to maintain the system; • operational faults that occur during service delivery of the use phase. 2. The location of the faults with respect to the system boundary: • internal faults that originate inside the system boundary; • external faults that originate outside the system boundary and propagate errors into the system by interaction or interference. • natural faults that are caused by natural phenomena without human participation; • human-made faults that result from human actions. Taxonomy   9 4. The dimension in which the faults originate: • hardware (physical) faults that originate in, or affect, hardware; • software (information) faults that affect software, i.e., programs or data. • malicious faults that are introduced by a human with the malicious objective of causing harm to the system; • non-malicious faults that are introduced without a malicious objective. 6. The intent of the human(s) who caused the faults: • deliberate faults that are the result of a harmful decision; • non-deliberate faults that are introduced without awareness. 7. The capacity of the human(s) who introduced the faults: • accidental faults that are introduced inadvertently; • incompetence faults that result from lack of professional competence by the authorized human(s), or from inadequacy of the development organization. 8. The temporal persistence of the faults: • permanent faults whose presence is assumed to be continuous in time; • transient faults whose presence is bounded in time. If all combinations of the eight elementary fault classes were possible, there would be 256 different combined fault classes. In fact, the number of likely combinations is 31; they are shown in Figures 3.3 The combined faults of Figures 3.3 and 3.4 are shown to belong to three major partially overlapping groupings: • development faults that include all fault classes occurring during development; • physical faults that include all fault classes that affect hardware; • interaction faults that include all external faults. The boxes at the bottom of Figure 3 .3 identify the names of some illustrative fault classes. The definition of human-made faults (that result from harmful human actions) includes absence of actions when actions should be performed, i.e., omission faults, or simply omissions. Performing wrong actions leads to commission faults. The two basic classes of human-made faults are distinguished by the objective of the developer or of the humans interacting with the system during its use: • malicious faults, introduced during either system development with the intent to cause harm to the system during its use (#5-#6), or directly during use (#22-#25) Dependability and Its Threats: A Taxonomy • non-malicious faults (#1-#4, #7-#21, #26-#31), introduced without malicious objectives. Malicious human-made faults are introduced by a developer with the malicious objective to alter the functioning of the system during use. The goals of such faults are: (1) to disrupt or halt service, (2) to access confidential information, or (3) to improperly modify the system. They are grouped into two classes: • potentially harmful components (#5, #6): Trojan horses, trapdoors, logic or timing bombs; • deliberately introduced software or hardware vulnerabilities or humanmade faults.. The goals of malicious faults are: (1) to disrupt or halt service (thus provoking denials-of-service), (2) to access confidential information, or (3) to improperly modify the system. They fall into two classes: Deliberate, non-malicious, development faults result generally from tradeoffs, either a) aimed at preserving acceptable performance, at facilitating system utilization, or b) induced by economic considerations. Deliberate, non-malicious interaction faults may result from the action of an operator either aimed at overcoming an unforeseen situation, or deliberately violating an operating procedure without having realized the possibly damaging consequences of this action. Deliberate non-malicious faults share the property that often it is recognized that they were faults only after an unacceptable system behavior, thus a failure, has ensued; the developer(s) or operator(s) did not realize that the consequence of their decision was a fault. It is often considered that both mistakes and bad decisions are accidental, as long as they are not made with malicious objectives. However, not all mistakes and bad decisions by non-malicious persons are accidents. Some very harmful mistakes and very bad decisions are made by persons who lack professional competence to do the job they have undertaken. A complete fault taxonomy should not conceal this cause of faults, therefore we introduce a further partitioning of both classes of non-malicious humanmade faults into (1) accidental faults, and (2) incompetence faults. The structure of this human-made fault taxonomy is shown in Figure 3 .5.
According to Avizienis et al. REF dependability is a superordinate concept regrouping different system attributes such as reliability, safety, security, or availability and nonfunctional requirements for modern embedded systems.
Dependability and Its Threats: A Taxonomy
IFIP Congress Topical Sessions
Computer Science