Knowledge-Based Evaluation

This article investigates the implications on formal evaluation theory when existing methods are viewed from an epistemic perspective. A case is made that from this approach a meaningful evaluation might be made with only a shared justification system present. Items of Truth and Belief, although of extreme interest to the parties of most evaluations, are shown to be irrelevant to the advancement of knowledge that may come about as a result of the evaluation itself. The role of this system of justification is further expanded to literally define what a knowable item within the context of an evaluation might be. These ideas are particularly explored within the contexts of emerging systems and domains of inquiry. ********** From January 13-15th, 1999 I had the opportunity of attending a series of meetings on continuing accreditation sponsored by the National Council for Accreditation of Teacher Education (NCATE). During these meetings, numerous evaluation problematics were discussed and I found myself surrounded by evaluation issues within the performance assessment arena. Some of these included how one identifies the nature of that which is to be evaluated, how it might be done. In particular, one issue kept coming to my mind regarding the artificial distinction which is faced in nearly every evaluation between how one records the evaluation process itself as opposed to merely recording the products that are to be evaluated. This directly stimulated this discussion. As I examined the working definitions for the terms knowledge and understanding we were provided with in these sessions, I found them nearly opposite from the same definitions when defined from a traditional epistemic perspective. As presented in the workshops, the best descriptors I could get were: Knowledge--a threshold awareness of, and Understanding--a more comprehensive, thorough level, which permits interpretation of the content in each standard. These differences go beyond semantics and have far reaching implications that make them far from a trivial matter. If knowledge is to be equated with mere awareness it should come as no surprise that many participants at this NCATE session came to feel that the evaluation, and subsequent report of the evaluation, can and should be expressed in concrete and easily documented artifacts. A simple checklist, as it were, could serve as a knowledge proof. If we adopt a more robust view of knowledge, however, a checklist approach is clearly inadequate. Let us take the position that knowledge, whatever else it may entail, cannot be captured by a simple awareness--but is rather a network of relationships between justification systems, truth, and belief. This notion is not unique to this article, and indeed serves as a fairly traditional component of many epistemological discussions (Whitehead, (1978) for one such discussion). For the sake of argument, let us propose the following working definition for knowledge: K = J T B where K refers to Knowledge, J to Justification, T to Truth, and B to Beliefs. Let us further imagine that as a starting point that these terms are in their perfect ideal state and are totally understood in their entirety by all potential knowers. A bit ambitious premise, but at least we are advancing from mere awareness. It is clear that this defintion will not hold up for long. In the post-modern world in which we live most people no longer accept the idea of a universal Truth. Truth is viewed, for the most part, as consisting of sets of contextually dependent truths. When we adapt our traditional knowledge equation to reflect this scenario we could define some locally dependent truths (as opposed to Truth) which best reflects the current state-of-the-art within the field. So now we have: K = J t B As might be guessed, this is not the only compromise that must be made in this ideal formulation. …