The Rashomon Effect

Rain. Torrential rain. Ceaseless rain, washing down in sheets so thick that fire hoses seemed to be trained on the decaying remnants of the Rashomon gate. Huge drops so opaque that they might have been tinted with black ink. At least in this instance, appearance is in fact reality, because that’s exactly how Akira Kurosawa created the spectacularly inclement weather for his legendary 1950 film, Rashomon. The tale begins with the sight of a peasant desperately fleeing the watery onslaught for the small patch of shelter offered by the ruins of the city’s southern gate. Rashomon’s once heroic grandeur has now become an unsavory hideout for thieves, a convenient place to abandon anonymous corpses and unwanted infants. The two men the peasant finds there are in a state of shock; a priest so stunned by the depravity of what he has seen that he has almost lost his faith in humanity; a woodcutter so profoundly disturbed that he must share his story, welcome or not. The woodcutter relates that three days earlier he had the misfortune to discover the grisly remains of a samurai in a remote spot in the forest. A delicate woman’s hat and some knotted ropes found at the scene could only hint at what might have occurred. The notorious bandit Tajomaru was quickly captured and confessed to waylaying the samurai and raping his wife. The woodcutter and the priest attended the bandit’s subsequent trial. Although the basic facts were uncontested, each of the three witnesses at the tribunal—Tajomaru, the samurai’s wife, and, speaking through a medium, the dead man himself—gave a strikingly different account of the heinous crime. Each of the three, in turn, took responsibility for the death; yet cast it in a light that made their own behavior appear to have been honorable. After these three different versions are retold, the woodcutter confesses that he actually witnessed the entire crime but did not admit it at the trial because he did not want to get involved. In his contrasting description, none of the three participants behaved very nobly. The veracity of even this seemingly impartial account is questioned by the peasant. Kurosawa accomplishes his narrative so ingeniously that what ‘‘really’’ occurred is never entirely certain; the viewer is left to contemplate multiple descriptions of reality that cannot be completely reconciled. Kurosawa’s approach to storytelling through differing subjective versions proved so powerful a device that it has since been imitated in countless motion pictures and television programs. Applications of this concept have also found relevance beyond the dramatic arts. Karl Heider, an ethnographer, applied the rubric ‘‘the Rashomon effect’’ to cases in which different anthropologists describe the same society in vastly discrepant terms, although they all presumably utilized the same scientific methods in their analysis. While we may not always recognize it, the Rashomon effect can occur in orthopaedic sports medicine as well. Different experts may apply the same seemingly objective evaluation criteria and yet produce radically differing results. Two studies in this month’s American Journal of Sports Medicine examine the capacity of multiple evaluators to employ imaging standards with consistency in the assessment of rotator cuff disease. Both of these studies tested the ability of expert observers to sort patient data into commonly utilized diagnostic categories. Both report the success of the observers using a statistical method proposed by Jacob Cohen in 1960, the kappa statistic (k). As Cohen noted, ‘‘The most primitive approach has been to simply count up the proportion of cases in which the judges agreed and let the issue rest there. ’’ The limitation of this basic method is that a certain amount of agreement will occur by chance alone. In the simplest case, if two observers randomly allocate data between two categories, they will make the same choice 50% of the time. As the number of observers and possible categories increases, the probability of random agreement will decline to varying degrees. For a classification system to be meaningful, one would want the prevalence of agreement to be substantially beyond that attributable to pure chance. To express this concept, Cohen proposed k ‘‘to represent the proportion of agreement after chance agreement is removed from consideration. ’’ In Cohen’s system, k can vary between 11 and 21, with 11 signifying perfect agreement, 0 the amount of agreement attributable to pure chance, and negative values representing agreement worse than that expected from chance alone. Further embellishments of Cohen’s basic concept have followed. Cohen’s original k, the one usually seen in orthopaedic reports, assumes that all disagreements are equally serious. Cohen later introduced the weighted k, which allows for the specification of some disagreements as more undesirable than others. In 1977, Landis and Koch arbitrarily divided k values into 6 grades of increasing desirability. In their classification system, which is widely quoted, negative k values are considered poor agreement, while positive values are partitioned into 5 equally sized categories of slight (0.00-0.20), fair (0.210.40), moderate (0.41-0.60), substantial (0.61-0.80), and almost perfect (0.81-1.00). Editorial