Source: http://www.apdex.org/index.php/2010/07/which-apdex-features-can-be-generalized/
Timestamp: 2019-04-18 10:59:23+00:00

Document:
Which Apdex Features Can Be Generalized?
I’m writing a series of posts about Generalizing Apdex. This is #6.
The current Apdex specification is entirely focused on application response time, specifically on the response time of Tasks and Task Chains. However, people have already adopted the Apdex method as a convenient way to report on other metrics, including network “turns”, bandwidth, and VOIP quality scores. These adaptations demonstrate the strength and adaptability of the core Apdex concepts. But because such ad hoc extensions fall outside the standard, they are likely to be implemented in inconsistent ways. By generalizing the Apdex spec, we aim to rectify that situation, bringing a wide class of metrics in a wide range of measurement domains within the scope of the standard.
Which Apdex features are candidates to be generalized? To answer that question, I will review the current Apdex specification. For clarity, I will use the section symbol (§) when referring to specific sections or subsections of the spec. During this pass, my goal is only to identify the major aspects of the current standard to be reviewed and reworked, not to address the actual structure or language of the generalized spec. It will be much easier to produce precise language once the concepts are clear.
To simplify the discussion here, I will also introduce letter suffixes to denote specific elements of an Apdex spec. This idea is modeled on the naming scheme used by the Transaction Processing Performance Council for their TPC benchmarks.
This naming scheme, and my particular choices of letter suffixes, are arbitrary editorial conveniences at this point; they have never been discussed by the Alliance. However, there may be marketing benefit from defining different versions of Apdex in this way. The existence of separately identified versions of the Apdex standard may help to emphasize its broad applicability. If you have opinions or alternative suggestions as to the best way to name and distinguish the components of a generalized Apdex spec, please contribute them in a comment to this post.
While it can be reasonably applied to many metrics in many measurement domains, there are probably some in which the Apdex method would be more applicable than the name itself. Does that mean we should not seek to apply Apdex in those domains? I don’t think so; it’s sufficient that the name reflect the history of the method. In practice this may not be a real issue, but the origin of Apdex should be noted somewhere in the introduction to Apdex-G.
[§2.1] Earlier posts in this series on Core Apdex Qualities and Apdex as a (Key) Performance Indicator have discussed the subject of Index Objectives.
[§2 (preamble), §2.3, §2.4] These sections are specific to measurements of application responsiveness; they belong in Apdex-R.
The present spec (§2.5) contains the rule that the value of F is four times the value T. This rule of thumb, while convenient, is not generally applicable outside the application responsiveness domain; it must be moved to Apdex-R. Since F cannot, in general, be derived from T, Apdex-G must permit both T and F to be specified separately, and make no assumptions about which is the larger value. In some measurement domains, larger values are more desirable.
Separately Configurable Thresholds will be the subject of a future post.
These changes are unlikely to encounter any objections. They have been proposed for several years within the Apdex community, and are already employed in some Apdex-based reporting. For example, Keynote Systems implemented them in their VOIP quality reports. See Clarity in Voice Performance Measurements with Apdex, presented by Ken Harker at the Apdex Symposium in December 2007 [PPT file].
To accommodate these cases, we would need Apdex-G to provide more flexibility when configuring targets and zones. See also “Configurable Scoring” under §4 below.
Configurable Zone Alignment will be the subject of a future post, in which I will aim to address all requirements for separately configurable threshold values.
[§2.6] How the Index Works: This needs minor wording changes to remove any references to ‘seconds’.
[§3.1] Response Time Measurements: most of this section must be moved to Apdex-R. It can be replaced by more general guidelines about measurement tools.
[§3.3] The entire discussion of Thresholds must be generalized, removing references to ‘seconds’ and incorporating any decisions made about “Configurable Zone Alignment” (§2.5).
[§3.4] Number of Measurement Samples: The core ideas in this section can remain, with minor wording changes. However, the minimum number of samples is probably domain-specific, so that rule should be moved to the addenda. Apdex-R should specify a minimum of 100 samples.
The present spec defines an attractively simple scoring rule in which measurements falling within the zones Satisfied, Tolerating, and Frustrated receive scores of 1, ½, and 0 respectively. Alan Ackers and Neil Gunther have discussed possible enhancements that would involve retaining the core approach to scoring while allowing for more than two targets (or thresholds), thereby creating more than three zones, and using finer scoring gradations (but still between 0 and 1) for rating measurements that fall within those zones.
The changes necessary to permit these kinds of generalization are closely related to those discussed under “Configurable Zone Alignment” (§2.5) and the two topics must be considered together.
Configurable Scoring will the subject of a future post.
[§4.1] The description of the Apdex Formula must be rewritten to incorporate any decisions made about “Configurable Zone Alignment” and “Configurable Scoring”.
[§4.2] This discussion of Exceptions must be moved to Apdex-R. I’m not sure if we need to replace it with any more general equivalent.
[§5.1, §5.2, §5.3] The rules for Displaying the Apdex Value and Indicating Sample Size must be generalized to handle “Separately Configurable Thresholds” (§2.5). Also, these requirements of the present Apdex standard may need to be modified to allow Apdex to be applied in many other domains.
The issues to be considered in the generalization process are (1) the degree of generalization to be accommodated, (2) the notation to be used to describe the more general targets, zones, and scoring, and (3) because the more general notation will necessarily be more complex, whether to retain any simpler notation for use in simpler cases like those defined in the present spec. This is an area for further research; I will try to identify some specific examples of the requirements of other domains.
Configurable Reporting will the subject of a future post.
The thresholds defining the Apdex Rating Bands (Excellent, Good, Fair, Poor, Unacceptable) are already defined as optional in section 5.4 of the present spec. But if implemented, the details must follow the specification.
According to Jonathan Becher, such grades are an essential feature of a KPI. However, I suspect that the Apdex thresholds, and the names of the five rating bands they define, may need to be configurable so that an Apdex implementation can conform to existing practice within a measurement domain, or within a user organization.
Configurable Rating Bands will the subject of a future post.
We should review, and decide what is appropriate as the Apdex-G spec evolves. One possibility is to convert this section into a “History” section, unless we have references that support the concepts behind the generalized version of Apdex. There is some flexibility in the choice and allocation of explanatory background material to sections §1, §2, and §6, so I intend to defer decisions on the exact composition of those sections until later in the process.
See my previous post, An Extensible Apdex Glossary. My current view is that all Apdex spec documents should share a common glossary; any other approach seems likely to be a maintenance headache.
These posts will discuss Apdex-G at the next level of detail, and I will link each post as it is published. Otherwise I do not plan to extend or refine this post unless I discover an error or omission. However, my conclusions in this post are certainly open for public comment. If you post a question or suggestion below, I will respond.

References: §2
 §2
 §4
 §5
 §5
 §1
 §2
 §6