Intuiting the Cognitive Line in Developmental Assessment: Do Heart and Ego Develop Through Hierarchical Integration?

existing cognitive content, is easily pictured as adding a new layer of neurons or connections that sense, measure, react to, coordinate or regulate a prior or "lower" layer. 11 I spent decades working in areas of applied artificial intelligence where I was exposed to many theories of formalizing human knowledge, skill, and behavior; and to the progression of academic expectations in these areas as scholars became ever more aware of the true complexities of human behavior (and living systems in general, including animal behavior and social systems). This, in addition to my own work in "knowledge representation" and "knowledge acquisition," plus studies in psychology and philosophy, have made me acutely aware (overly sensitive? ) of the limitations in our attempts to formalize complex natural systems. 12 Of course, there are uncertainties in our claims about statistical uncertainty, and the issue is recursive in a never-ending way. But just going this one level deeper provides significant value (going two or three levels deeper becomes so difficult to conceptualize that there are diminishing returns). 13 Lakoff and Johnson (1999) maintain that the constructs available to us from which to build our models (abstract concepts, structures, operators, etc.) are deeply metaphorical. These metaphorical building Murray: Intuiting the Cognitive Line in Developmental Assessment INTEGRAL REVIEW December 2009 Vol. 5, No. 2 349 That is, we know that conceptual categories and the models and theories that are build from them inevitably have fuzzy boundaries and diverse interpretations, and we know that our perceptions, abstractions, and generalizations are limited by various sorts of biases and distortions; lenses and filters. But we often share claims as if we did not know about these limitations. Indeed, in abstract (conceptual and non-measurable) domains it is not an easy thing to do—it takes tools we are historically just beginning to become acquainted with and effort that is in short supply to step back and evaluate the limits of one’s claims with any sort of rigor. And the norms of modern scholarly discourse give little incentive to do so. But I propose that as we become more comfortable and familiar with the nature of various sorts of indeterminacy (including what Fisher and Stein (2008) have called "dark knowledge"), through findings in psychology and philosophy, and increase our familiarity with this terrain, we can expand meaning-generation potential and efficiency in a community of inquiry. This allows for more nuanced dialog. For example, the concepts of mind and consciousness are quite problematic in their definition, yielding various types of arguments about their nature which are highly fallible and vulnerable to counterargument. Perusing the debated claims and counterclaims in the literature is dizzying. From the perspective of am interested reader not attached to a particular answer (i.e., myself) it seems as though so much ink is wasted when authors hold onto a particular definition and do not yield to the possibility of alternate perspectives. Another example is in the use of theoretical models such as AQAL. As Wilber is quick to note, AQAL is a map and not the territory itself—it is a particular way of cutting up the infinitely complex, nuanced, and ineffable territory(s) of reality, useful for a particular set of purposes. This noting constitutes the type of humility I mentioned above. But the next step, not so often taken, is to say as much as one can about the nature of this way of slicing up the "pie" such that we can understand the limitations of this powerful conceptual tool. What does it assume that might not be true? What admits to alternative views? What types of phenomena are not adequately captured using this lens? What things fall into the fuzzy or gray areas between the crisp conceptual lines of the four quadrants (and the other categories such as states vs. stages)? My argument is that all models and theories would be more adequate if they were described and "delivered" with a meta-wrapper or boundary analysis explaining some of what is known about the limitations of the "map. " How does this relate to our current topic of developmental assessment? Developmental assessments, if approached from the perspective of pure metrics, are subject to uncertainties akin to the right-hand-side realm of statistics, and their indeterminacy is thus more understood and less problematic. But to relate a pure metric to anything of relevance, to say what its results mean, we rely on models (or other tools of interpretation). So we are back to the limitations of models. blocks are both tools and lenses that reflect the sensory-motor (and other) primitive elements of our experience, as much as they reflect truths about the world. 14 For example, rather than "you are wrong" the dialog becomes more like "if one interprets the concepts [e.g., consciousness] in [. this. ] way there is validity to what you say, but my focus leads me to interpret it [consciousness] in [. this. ] way, which I claim is more useful. " 15 Also, assessments are designed in the first place to serve the needs of a model or theory, so reevaluation of the nature of a model may lead to reevaluation of the nature of a related metric. Murray: Intuiting the Cognitive Line in Developmental Assessment INTEGRAL REVIEW December 2009 Vol. 5, No. 2 350 The models being developed using constructs such as hierarchical complexity and hierarchical integration are proving to be extremely powerful in uncovering deep structure, and helping explain and perhaps even predict developmental phenomena. This is in part due to their additional levels of formality. Increasing theoretical formality involves, among other things, making sharper conceptual cuts in the fabric of observed reality. The benefits of doing so are clear, as can be seen in the contributions of modern science and technology. However, in this post (or post-post) modern era, we are also increasingly aware of the limitations of formal systems to capture truths about reality. And, importantly, are beginning to become aware of the shape of or nature of these limitations (as in the uncertainty principle and Gödel's incompleteness theorem), in such a way that we can work with the uncertainty, not just be helplessly subject to it. Though I have some incompletely-worked-out ideas about what this might look like, for this informal article I will simply suggestion that we put more effort into such "boundary analysis" as a way to compare, contrast, and combine the best of the current developmental models and assessment methods. Also, I should note that, though each formal model has its limitations, particular practitioners or projects sometimes use a combination of tools, each of which may compensate for the limitations of the others. For example, Dawson's (Dawson & Stein, 2008) "developmental maieutics" is about embracing multiple approaches. The Landscape of Assessment Tools Up to here my discussion has brought together ideas seen elsewhere in an attempt to help clarify and promote dialog and meta-reflection on the practices of making and using developmental models and instruments. Next I will slip into much more speculative territory, with the hopes that my intuitions and musings will lead to productive dialog rather than simply expose the limits of my experience in this domain (! ). The current landscape of developmental tools includes those that seem to have had very little if any assessment of validity, those, such as Cook-Greuter's and Torbert's LDP-related instruments, that have shown some validity (but perhaps do not raise to the standard of calibrated measures, as called for by Stein and Heikkinen), and those, including the LAS, that have shown 16 Of course, within the power of this new type of knowledge we are best maintain humility, as the recursive questions about indeterminacy have never ending depth. 17 These include many that are used in leadership and organizational development, too numerous to list here or single any out (but see Stein and Heikkinen's article in IR's prior issue). Presumably those using them believe that the value added in terms of generating meaning for clients outweighs the problems inherent in their lack of validity. I am very sympathetic with the cautions brought out by Stein and Heikkinen (prior issue), but hold out the possibility that in some cases soft metrics can be used in such a way that there is overall value added, for example in guided self-evaluation (and see the discussion of first, second and third person methods in Torbert, this issue). I am also sympathetic to the pragmatic reality that if an organization has found benefit for self and client in a "tried-and-true" instrument, there may be little incentive, nor even the realistic possibility, for them to fund the type of large scale research project needed to sufficiently validate the instrument. The potential to harm clients is very real and must be looked at squarely in each case. But it would be an overstatement to say that all non-validated instruments are harmful. Murray: Intuiting the Cognitive Line in Developmental Assessment INTEGRAL REVIEW December 2009 Vol. 5, No. 2 351 strong (calibrated) validity and are also based on a deep/formal developmental theory. It is the latter two categories that I am concerned with here. My understanding is that Stein and Heikkinen (and others in the neo-Piagetian line) hypothesize that their formal frameworks could measure much of what the LDP and related frameworks are measuring, and do so in a more rigorous fashion; and that some of those using LDP-related tools believe that they are able to assess something that the more formal models do not (or cannot). Both "camps" if I can call them such, agree that the ultimate answer to these questions will come only through empirical work in comparing the systems (as Heikkinen (this issue) is doing now between the LAS and Ke