Potential Role of Anti-GAD Antibodies in Abnormal Eye Movements ABSTRACT: Glutamic acid decarboxylase (GAD) catalyzes the conversion of glutamic acid to ␥-aminobutyric acid (GABA). Autoantibodies directed against GAD (antiGAD-Ab) have been described in patients with insulin-dependent diabetes mellitus, stiff-man syndrome, and in a few patients with progressive cerebellar ataxia. The presence of these autoantibodies suggests an autoimmune pathophysiological mechanism for the neurological manifestations in these disorders. However, the exact role of antiGAD-Ab and GABAergic neurotransmission in the pathogenesis of the neurological manifestations, particularly in progressive cerebellar ataxia, is not fully understood. The cases of two patients with subacute cerebellar ataxia associated with antiGAD-Ab presenting with abnormal eye movements are reported. One patient presented a periodic alternating nystagmus (PAN), whereas the other presented a downbeat nystagmus (DBN) and slow vertical saccades. The potential role of antiGAD-Ab and the resultant GABAergic neurotransmission deficit in oculomotor manifestations is discussed. Glutamic acid decarboxylase (GAD) is a major enzyme in the nervous system that catalyzes the conversion of glutamic acid to γ-aminobutyric acid (GABA). GABA is a major inhibitory neurotransmitter in the nervous system, but also is present within the β cells of the pancreas. GAD has long been identified as an essential autoantigen in the development of stiff-person syndrome and insulin-dependent diabetes mellitus. [1] [2] [3] [4] Anti-GAD antibody (antiGAD-Ab) has more recently been identified in association with late-onset cerebellar ataxia. [5] [6] [7] [8] The specific and pathogenic role of antiGAD-Ab in neurological disorders such as cerebellar ataxia is not fully understood. 9, 10 These autoantibodies could be an epiphenomenon caused by destruction of the nervous system. However, the increased association with other autoimmune manifestations, as well as evidence for intrathecal inflammation, suggest an autoimmune-mediated cerebellar impairment. In the case of a direct role of antiGAD-Ab in this disease, can we provide evidence that neurological manifestations are related to a functional deficit in GABAergic neurons?. We report on two patients presenting with late-onset cerebellar ataxia and abnormal eye movements associated with the presence of anti-GAD antibodies. We suggest that precise knowledge of the pathophysiology of abnormal eye movements in these cases may help to better understand the pathophysiology of neurological symptoms associated with antiGAD-Ab. Moreover, the occurrence of abnormal eye movements with antiGAD-Ab antibodies may also help to better understand the neurochemistry of the eye-movement neural network. One of the two cases has been reported previously. 11 We report the cases of two patients showing different oculomotor manifestations in association with cerebellar ataxia and anti-GAD antibodies. The main oculomotor manifestation in Case 1 was a periodic alternating nystagmus, whereas Case 2 presented with downbeat nystagmus (DBN) and slowed or absent vertical saccades. In the following, we discuss the potential involvement of antiGAD-Ab and the resultant GABAergic neurotransmission deficit in these oculomotor manifestations.. As previously discussed, 11 PAN in Case 1 provides the opportunity to invoke the direct implication of antiGAD-Ab in cerebellar dysfunction. In man and monkeys, PAN has been reported with lesions of the caudal and medial parts of the cerebellum, involving mostly the nodulus and uvula. 12, 13 The nodulus and uvula appear to control the time course of the postrotational vestibulo-ocular response (VOR), leading to prolonged VOR, as found in our patient. Normal vestibular repair mechanisms act to reverse the direction of the nystagmus, causing periodic oscillations of the nystagmus. 13, 14 Such oscillations would normally be suppressed by visual fixation. In our patient, poor visual acuity due to age-related macular degeneration, as well as floccular involvement, known to control smooth pursuit and fixation, might explain the occurrence of the nystagmus. Pharmacological evidence suggests that the maintenance of nodular and uvular inhibitory control on the duration of vestibular rotational responses is mediated by GABA. 15 Indeed, in our patient, as in most patients with acquired PAN, the GABAergic drug, baclofen, abolishes nystagmus. 16, 17 Therefore, in Case 1, we suggest that acquired PAN is linked to a functional GABAergic deficit of cerebellar pathways that can be alleviated by the GABAergic drug baclofen.. DBN is the most frequent oculomotor manifestation associated with progressive cerebellar ataxia. 18 It has been attributed to lesions involving the posterior cerebellum, especially the flocculus and paraflocculus. 19 The exact pathophysiological mechanism of DBN is not yet known. The more widely accepted explanation currently is an asymmetry in the activity of vertical slow-phase pathways (see Ref. 20 for review). Whether this asymmetry involves a vertical vestibulo-ocular, smooth pursuit, or velocity-to-position integrator has not been determined yet. However, it is widely assumed that this tonic asymmetry is nearly perfectly suppressed by a posterior cerebellar central mechanism, but is released from inhibition when these cerebellar structures are lesioned. 20 In our patient, the presence of abnormal horizontal and vertical smooth pursuit and VOR suppression support the hypothesis of floccular involvement. A case of isolated downbeat nystagmus associated with antiGAD-Ab has been reported recently. 21 Both these cases suggest GABAergic neurotransmission involvement at the floccular level. Absence of drug response (baclofen) might be due to the more complex neural circuitry for the control of vertical slow-phase eye movements.. Both PAN and DBN could be explained by GABAergic neurotransmission impairment at the Purkinje cell level. Case 2 presented with abnormal vertical saccades, suggesting a specific involvement of upward vertical saccade pathways. Although much has been learned about the anatomy and physiology of the saccadic system, the pharmacology is still being elucidated. Slow saccades are best explained by lesions involving the excitatory burst neurons or omnipause neurons. 22 In order to explain isolated slowness or absence of upward vertical saccades, involvement of the rostral interstitial nucleus of the medial longitudinal fasciculus (riMLF), where burst neurons for vertical and torsional saccades are located, might be suggested. However, looking for GABAergic neurons in the rostral mesencephalon, Horn et al. could not show direct inhibitory GABAergic neurons from the riMLF to the oculomotor nucleus. 23 On the other hand, these authors found direct GABAergic projections from the interstitial nucleus of Cajal (INC) to the oculomotor neurons that could be involved in the inhibition of premotor down burst tonic neurons during upward eye movements. In Case 2, a GABAergic neurotransmission deficit due to antiGAD-Ab could alter the function of these inhibitory neurons and the execution of upward saccades. A precise analysis of the velocity waveform of upward saccades could help to demonstrate abnormal braking instead of general slowness of vertical saccades. Other unknown GABAergic saccadic inhibitory pathways might also be involved in the upward vertical saccade deficit in Case 2.. In conclusion, we report the cases of two patients in whom nystagmus is probably explained by GABAergic neurotransmission deficits. These observations suggest a direct pathogenic role of antiGAD-Ab in the occurrence of neurological manifestations. Given such a hypothesis, the occurrence of other abnormal eye movements, such as the abnormal upward saccades in Case 2, may help to better understand the neurochemistry of the eye-movement neural network. However, questions remain unsolved, such as the diversity of clinical manifestations and the variable efficiency of GABAergic drugs. This heterogeneity of antiGAD-Ab syndromes may be related to different antigenic targets, an aspect that remains to be resolved. Results of eye-movement recording in the control group and the two patients are summarized in TABLE 1.

O O O O O O O O O O B I I I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O B O O O O B O O O O B O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On Successive Refinement for the Kaspi/Heegard-Berger Problem Abstract-Consider a source, {Xi, Yi, Zi , producing independent copies of a triplet of jointly distributed random variables (RVs). The {Xi} part of the process is observed at the encoder, and is supposed to be reproduced at two decoders, decoder 1 and decoder 2 , where the {Yi} and the {Zi} parts of the process are observed, respectively, in either a causal or a non-causal manner. The communication between the encoder and the decoders is carried in two successive communication stages. In the first stage, the transmission is available to both decoders and the decoders reconstruct the source according to the received stream and individual side information ({Zi} or {Yi }). In the second stage, additional information is sent to both decoders and the decoders refine the reconstruction of the source according to the side information available to it and the transmissions in both stages. It is desired to find the necessary and sufficient conditions on communication between the encoder and decoders, so that the distortions incurred (at each stage) will not exceed given thresholds. For such a multidecoder coding setting with successive refinement and non-causal degraded side information, we derive inner and outer bounds to the achievable rate-distortion region. Then, for the case of general causal side information at the decoders, we derive a single-letter characterization of the achievable region for a multidecoder source-coding problem with successive refinement. In this paper, we consider an instance of the problem of multiple description, which is successive refinement of information. The term successive refinement of information is applicable in systems where source reconstruction is done in a number of stages. In such systems, a source is encoded by a single encoder which communicates with either a single decoder or a number of decoders in a successive manner. In each stage, the encoder sends some amount of information about a source to a decoder of that stage which also has access to all the previous transmissions. The decoder bases its reconstruction on all transmissions available to it and, if available, on additional side information (SI). The quality of reconstruction in each stage (at each decoder) is measured with respect to a predefined distortion measure. In the case of pure source-coding, the information transmitted by the encoder at each stage arrives at the decoder noiselessly, while in the case of noisy channels connecting the encoder and decoders, the transmission arrives to the decoder corrupted and thus, joint source-channel coding should be applied.. A number of works treated the problem of successive refinement [1] - [4] , and a related problem of hierarchical coding [5] - [7] . In [4] , the problem of successive source coding was studied for the Wyner-Ziv setting, i.e., when SI is available to each decoder non-causally [8] . The encoder transmits a source sequence, X, to two decoders in two successive stages. Necessary and sufficient conditions were provided in [4] , in terms of single-letter formulas, for the achievability of information per-stage rates corresponding to given distortion levels of each communication step. For the case of identical SI available at all decoders, the two-stage coding scheme was extended to include any finite number of stages. Finally, conditions for a source to be successively refinable with degraded SI were introduced in [4] for the two-stage scheme. Generally speaking, the notion of degraded SI means that the quality of SI available at the decoders of later stages is better than that of earlier stages.. In [6] , the problem of successive refinement with SI available non-causally at each decoder was studied from a different viewpoint. Instead of considering per-stage communication rates, the analysis of successive refinement was performed with respect to cumulative (sum-) rates achievable at each stage, under per-stage source restoration assumptions. A single-letter characterization of the achievable region with successive coding sum-rates and distortions was provided for the case of degraded SI at the decoders. It turned out that when the rate-sums are analyzed, it is possible to characterize an achievable rate-distortion region for any number of stages as long as the SI at the decoders is degraded.. In [7] , the problem of successive refinement was investigated for the case of SI available causally at the decoders. It turned out that, unlike the above-described non-causal settings, when SI is available causally, the characterization of the achievable per-stage rate-distortion region is possible without constraining SI to be degraded.. The works reported in the field of successive refinement thus far have considered refinement of information when the transmission at each stage has been addressed to a single decoder. It is well known that there are many applications where a single encoder conveys information to a number of decoders in a single transmission. The works of Heegard and Berger [9] and Kaspi [11] study independently the following scenario: a single encoder communicates via a single transmission with two decoders one of which accesses the transmission only, while the other has a non-causal access to some SI correlated with the source. The source sequence should be reconstructed at both decoders with a certain accuracy and, under these distortion constraints, it is desired to reduce the communication rate as much as possible.. The minimum achievable communication rate, i.e., the ratedistortion function obtained for this setup has been named the Heegard-Berger rate-distortion function. It was extended in [9] to include a coding scheme with more that two decoders, each having access to different SI's with a degraded structure. Now, assume that there is a demand for a better reconstruction at either one or both decoders, i.e., the source is required to perform a multi-level successive refinement, still communicating with all decoders via a single transmission. A question of obvious interest is the following: is it possible to characterize the achievable rate-distortion region for this generalized problem of successive refinement?. In this work, we extend the works of [9] , [4] , [6] and [7] . Specifically, we study the scenario of two-decoder, two-stage successive refinement of information, with SI available at all decoders in either a causal or a non-causal manner. For the former case, we provide a single-letter characterization of the achievable rate-distortion region, which is straightforwardly extendable to any number of decoders accessible in each stage and any finite number of stages. For the latter case of SI available at the decoders non-causally, we provide inner and outer bounds to the achievable rate-distortion region for the case of degraded SI. Note that SIs are no longer degraded in transition to the second transmission and therefore this setting is of particular interest. When considering the case of causal SI, we provide the achievable region in terms of the per-stage rates, while for the case of non-causal SI we refer to the sumrates. The difficulty in characterizing the per-stage rates for a general scheme here is similar to that faced in [4] .

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Integrating back, history and bookmarks in web browsers Most Web browsers include Back, History and Bookmark facilities that simplify how people return to previously seen pages. While useful, these three facilities all operate on quite different underlying models, which undermines their usability. Our alternative revisitation system uses a single model of a recency-ordered history list to integrate Back, History and Bookmarks. Enhancements include: Back as a way to step through this list; implicit and explicit 'dogears' to mark pages on the list (replacing Bookmarks); searching/filtering the list through dynamic queries; and visual thumbnails to promote page recognition.

O O B O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O B O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B


Bent and generalized bent Boolean functions In this paper, we investigate the properties of generalized bent functions defined on Z n 2 with values in Z q , where q ≥ 2 is any positive integer. We characterize the class of generalized bent functions symmetric with respect to two variables, provide analogues of Maiorana-McFarland type bent functions and Dillon's functions in the generalized set up. A class of bent functions called generalized spreads is introduced and we show that it contains all Dillon type generalized bent functions and Maiorana-McFarland type generalized bent functions. Thus, unification of two different types of generalized bent functions is achieved. The crosscorrelation spectrum of generalized Dillon type bent functions is also characterized. We further characterize generalized bent Boolean functions defined on Z n 2 with values in Z 4 and Z 8 . Moreover, we propose several constructions of such generalized bent functions for both n even and n odd. In the recent years several authors have proposed generalizations of Boolean functions [10, 15, 16, 18, 19] and studied the effect of Walsh-Hadamard transform on these classes. As in the Boolean case, in the generalized setup the functions which have flat spectra with respect to the Walsh-Hadamard transform are said to be generalized bent and are of special interest (the classical notion was invented by Rothaus [13] ).. Let us denote the set of integers, real numbers and complex numbers by Z, R and C, respectively and let the ring of integers modulo r be denoted by Z r . The vector space Z n 2 is the space of all n-tuples x = (x n , . . . , x 1 ) of elements from Z 2 with the standard operations. By '+' we denote the addition over Z, R and C, whereas '⊕' denotes the addition over Z n 2 for all n ≥ 1. Addition modulo q is denoted by '+' and it is understood from the context. If x = (x n , . . . , x 1 ) and y = (y n , . . . , y 1 ) are in Z n 2 , we define the scalar (or inner) product by x · y = x n y n ⊕ · · · ⊕ x 2 y 2 ⊕ x 1 y 1 . The cardinality of the set S is denoted by |S|, and the conjugate of a bit b will also be denoted byb. If z = a + b ı ∈ C, then |z| = √ a 2 + b 2 denotes the absolute value of z, and z = a − b ı denotes the complex conjugate of z, where ı 2 = −1, and a, b ∈ R.. We call a function from Z n 2 to Z q (q ≥ 2 a positive integer) a generalized Boolean function on n variables [16] . We denote the set of such functions by GB q n . If q = 2, we obtain the classical Boolean functions on n variables, whose set will be denoted by B n .. Let ζ = e 2πı/q be the complex q-primitive root of unity. The (generalized) Walsh-Hadamard transform of f ∈ GB q n at any point u ∈ Z n 2 is the complex valued function. If q = 2, we obtain the (normalized) Walsh-Hadamard transform of f ∈ B n , which will be denoted by W f . A function f ∈ GB q n is a generalized bent (gbent) function if |H f (u)| = 1 for all u ∈ Z n 2 . When q = 2, then f is bent (these exist for n even, only). If n is odd, a function f ∈ B n is said to be semibent if and only if |W f (u)| ∈ {0, √ 2}, for all u ∈ Z n 2 . Suppose f ∈ GB q n is a gbent function such that for every such u, we have H f (u) = ζ k u , for some 0 ≤ k u < q. Then, for such a gbent function f , there is a function F : Z n 2 → Z q such that ζ F = H f . We call such a function F the dual of f (Caution: only some gbent functions admit duals). By applying Theorem 1 below, one can easily see that the dual of a gbent function is also gbent, since the Walsh-Hadamard transform of the dual F is H F (u) = ζ f (u) . The sum. is the crosscorrelation of f and g at z. The autocorrelation of f ∈ GB q n at u ∈ Z n 2 is C f, f (u) above, which we denote by C f (u).. If 2 h−1 < q ≤ 2 h , for any f ∈ GB q n we associate a unique sequence of Boolean functions a i ∈ B n (i = 0, 1, . . . , h − 1) such that f (x) = a 0 (x) + 2a 1 (x) + · · · + 2 h−1 a h−1 (x), for all x ∈ Z n 2 .. If q = 4, then for f ∈ GB 4 n as in (1) we define the Gray map ψ( f ) : GB 4 n → B n+1 by ψ( f )(z, x) = a 0 (x)z + a 1 (x), for all (z, x) ∈ Z 2 × Z n 2 .. The function ψ( f ) is referred to as the Gray image of f .

O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O B I I O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Integrated Engineering Environments for Large Complex Products An introduction is given to the Engineering Design Centre at the University of Newcastle upon Tyne, along with a brief explanation of the main focus towards large made-to-order products. Three key areas of research at the Centre, which have evolved as a result of collaboration with industrial partners from various sectors of industry, are identified as (1) decision support and optimisation, (2) design for lifecycle, and (3) design integration and co-ordination. A summary of the unique features of large made-to-order products is then presented, which includes the need for integration and co-ordination technologies. Thus, an overview of the existing integration and coordination technologies is presented followed by a brief explanation of research in these areas at the Engineering Design Centre.A more detailed description is then presented regarding the co-ordination aspect of research being conducted at the Engineering Design Centre, in collaboration with the CAD Centre at the University of Strathelyde. Concurrent Engineering is acknowledged as a strategy for improving the design process, however design co-ordination is viewed as a principal requirement for its successful implementation.That is, design co-ordination is proposed as being the key to a mechanism that is able to maximise and realise any potential opportunity of concurrency. Thus, an agent-oriented approach to co-ordination is presented, which incorporates various types of agents responsible for managing their respective activities. The co-ordinated approach, which is implemented within the Design Co-ordination System, includes features such as resource management and monitoring, dynamic scheduling, activity direction, task enactment, and information management. An application of the Design Co-ordination System, in conjunction with a robust concept exploration tool, shows that the computational design analysis involved in evaluating many design concepts can be performed more efficiently through a co-ordinated approach. The Engineering Design Centre (EDC) at the University of Newcastle was established in 19y0, with support from industry and the Science and Engineering Research Council (SERC), now the Engineering and Physical Science Research Council (EPSRC). The initial focus of the research at the EDC was in the &dquo;marine and other large made-to-order (LMTO) products&dquo; sector, and this focus has remained throughout the centre's development. Industrial partners in the research programme are drawn from marine, offshore, chernical and process, power generation and aerospace sectors and include both suppliers and operators of equipment.. The main objectives of the research programme of the EDC are the development of tools and methods to improve the design process for LMTO products. The core of this programme is aligned around three principal themes, in a structure that has evolved in response to the research priorities set by the Centre's industrial partners. These themes are:~ Decision Support and Optimisatioñ Design for Lifecyclẽ Design Integration and Co-ordination This paper is primarily concerned with research in the third of these areas, &dquo;Design Integration and Co-ordination&dquo; which has been an active area of the research programme of the EDC for almost 10 years. The focus of this research is on the development of tools and processes to support the activity of large design teams that are organisationally and geographically distributed. While this is a general challenge in many modern engineering enterprises, it is particularly acute in the LMTO sector.. Even if discussion is confined to a particular sector, such as LMTO products, design integration and co-ordination is a broad field, which cannot be covered adequately in a short paper. This paper will therefore concentrate on the following principal topics: a summary of the unique features of LMTO products, and the resulting requirements for integration and co-ordination technologies a brief overview of integration and co-ordination technologies Xuihor to whol11 'orre~ponúel1l:e shttuld be addressed. · key features of the integration and co-ordination research projects currently underway at the EDC An individual research project within the integration and co-ordination portfolio at the EDC is beginning to be validated in an industrial scale case study.. Ever increasing competition in the global marketplace has forced companies to investigate new methods of improving product quality, lowering costs and reducing the time taken to introduce new products. This competitiveness makes constant improvement and modernisation crucial to survival. Since the late 1980s in academia and the early 1990s in industry, Concurrent Engineering (CE) has been recognised as one such strategy for improving the design process. This involves performing as many activities as possible simultaneously, enabled by cross-functional teams working cooperatively and effectively on separate aspects of the overall product development.. For a large engineering organisation, CE is seen by many as a requirement to achieve and sustain a competitive advantage in support of the development of high quality products that can be produced efficiently. CE involves varying sized groups of expertise working simultaneously on different parts of an engineering project. These teams are independent but cannot work in isolation since communication must occur between them. For complex product development, CE not only needs multi-functional teams, but also requires communication between teams and effective co-ordination to integrate their efforts. Hence, a communication environment is required to allow effective sharing of information between teams and among team members of related development tasks.. Many issues have been identified as essential requirements with regard to ensuring that CE is effective when implemented and operated in a large engineering organisation or complex design process. The most prominent issues of CE are co-ordination, communication, cooperation (teamwork), integration, information sharing, multi-functional teams, planning, scheduling, self-discipline and productivity [7] [8] [9] [10] [11] [12] .. It is argued that co-ordination is the principal requirement for the successful implementation of CE. Co-ordination can be thought of as the concept of the appropriate activities being performed, in a certain order, by a set of capable agents, in a fitting location, at a suitable time, in order to complete a set of tasks. That is, for the right reasons, at the right time, to meet the right requirements and give the right results 113]. The DCS has achieved significant reductions in the time taken to perform the computational analysis of a given number of design concepts within the RCE framework. Essentially, the time compression achieved by the DCS is inversely proportional to the number of processors utilised. As a result of significantly reducing the design analysis time, utilising the DCS has allowed the designer to perform a more comprehensive concept exploration of the design space. Consequently, the concept design selected was superior in terms of the designer's nominated criteria [20] .. It has been shown that the family of agents operating within the DCS can work cooperatively in a coordinated fashion with effective results. It is this ability of the agents to operate in a coordinated manner that permits the computational analysis time to be reduced by seizing the opportunity to perform activities concurrently. Simply committing greater resources to a particular part of the design process will not necessarily result in a corresponding reduction in the time to perform the tasks involved. It is the capacity to coordinate the activity performed by each of the team members, taking into account the available resources and knowledge of their roles and effects, and taking advantage of instances where concurrency is possible that enables a measured reduction in the duration of those activities to be achieved. Prior to discussing the results achieved in the catamaran design example, it is appropriate to mention the actions of the agents employed within the DCS. Once the RCE framework generated a number of alternative concept designs to be evaluated, the DCS was invoked. Initially, the Co-ordination Manager received notification of registration from all other agents, which included details of their attributes. Each agent then requested information from the Co-ordination Manager regarding related agents such that they could communicate directly thereafter. The Resource Manager was initially contacted by all Resource Monitors, which provided current measures of efficiency for the processor to which they were associated. Simultaneously, the Scheduling Agent constructed a task matrix, which held information regarding the tasks that need to be completed, i.e., the design concepts to be evaluated. Based on the information within the task matrix and resource model, the Scheduling Agent then employed the MCGA in order to determine a near-optimum order in which catamaran concept evaluations should be performed on the available processors taking into account their respective efficiency measures. The MCGA produced a Pareto optimal set of schedules, each of which was near optimal with respect to certain criteria. Minimise (I) time to complete the concept evaluations, (2) number of processors used, and (3) mean processor utilisation, were used as criteria to select the &dquo;best&dquo; schedule. The relevant part of this &dquo;best&dquo; schedule was then communicated to each respective Activity Director of the corresponding processor such that the analysis could commence. Activity Directors associated with each processor then requested that their related Task Manager of the corresponding design concept complete the concept evaluation. Prior to the completion of the concept evaluation, Task Managers requested that their related Information Manager provide the appropriate input information for the particular design concept to be evaluated. As soon as this information was supplier. the Task Manager completed the concept evaluation. Subsequently, the related Information Manager recorded the results of the evaluated design concept. The Activity Directors then continued directing the analysis in accordance with the schedule for their associated processor until the whole design analysis was completecl.. The measure of performance used for the DCS was the elapsed time to compute the solution of a given number of design concepts. In an environment with dedicated processors, and based solely on computation time of the design concepts, as the number of processors ii is increased the elapsed time should decrease according to the inverse relation llii.. Benchmark results were obtained using a single Ultra 1/170 machine to sequentially compute 79 and ?.~3 concepts for Case I and Case ? respectively. The results presented in Table 4 were obtained by running the DCS in comparable conditions for each operation with respect to processor utilisation. That is, it was ensured that other network usage was negligible.. Figures 4 and 5 indicate the total time taken for the family of agents operating within the DCS to perform their individual activities including the analysis tool execution duration for all design concepts being evaluated. Extrapolated benchmark execution times are also shown. The inverse relationship between the number of processors and computation time was used to obtain extrapolated results for 2, 3, 4 and 5 processors. Therefore, using ii processors as opposed to 1 should result in a 100(ii -I >fii% reduction in computational time. With respect to the extrapolated benchmark execution times, the assumption is made that for each combination of processors, all processors are working in parallel with an equal number of design concepts being evaluated sequentially on each of them.. Clearly, and as expected, in both Figures 4 and 5 it can be seen that as the number of processors is increased, the DCS execution duration decreases. The results obtained using the DCS appear to exhibit the previously stated inverse relationship with respect to the number of processors utilised. The results indicate that the DCS is capable of achieving results in close proximity to the projected benchmark values for both Case 1 and Case 2. With respect to Case 1, the offset between the DCS and benchmark results vary from approximately 12% to 5.5%. For Case 2 the offset varies from approximately 14% to 4.5%. In both cases, the greatest difference exists when only a single processor is employed within the DCS. This particular scenario is unrealistic since there would be no requirement for the DCS if only one processor were available as its intended use is in a distributed computing environment. The need for operating the DCS can only be justified in the event of more than one processor being available. After discarding the single processor use of the DCS, it can be deduced from Table 4 that a relatively small offset range of 7.2% to 4.5% envelops both cases. This proportion of DCS operation is due to activities performed by agents excluding analysis tool executions by Task Managers. Prior to any analysis tool execution, the Information Manager is tasked with preparing and coordinating the various design concept models throughout the distributed design environment. Similarly, evaluated concepts need to be coordinated for subsequent use, in this instance within the RCE framework, between analysis tool executions. In addition, certain activities of the Co-ordination Manager, Scheduling Agent, Resource Manager and Resource Monitors need to be performed before any analysis tool executions can commence.. In Table 4 it can be seen that a marginal difference in offset exists between Case 1 and 2. This difference is illustrated in Figure 6 . The discrepancy between the two cases can be explained by the fact that the proportion of coordination activity performed by the agents is less as a total percentage of the overall duration of the DCS in Case 2 than in Case 1.. Approximately three times as many design concepts were evaluated in Case 2 as in Case I and therefore analysis tool computation time becomes more dominant. Hence, the greater the number of design concepts being evaluated, the less the offset between the DCS and benchmark results.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O B I I O O O O O O O O B O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O B O B O O O O O O O O O O O O O B O B O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Source authenticity in the UMLS - A case study of the Minimal Standard Terminology As the UMLS integrates multiple source vocabularies, the integration process requires that certain adaptation be applied to the source. Our interest is in examining the relationship between the UMLS representation of a source vocabulary and the source vocabulary itself. We investigated the integration of the Minimal Standard Terminology (MST) into the UMLS in order to examine how close its UMLS representation is to the source MST. The MST was conceived as a "minimal" list of terms and structure intended for use within computer systems to facilitate standardized reporting of gastrointestinal endoscopic examinations. Although the MST has an overall schema and implied relationship structure, many of the UMLS integrated MST terms were found to be hierarchically orphaned, and with lateral relationships that do not closely adhere to the source MST. Thus, the MST representation within the UMLS significantly differs from that of the source MST. These representation discrepancies may affect the usability of the MST representation in the UMLS for knowledge acquisition. Furthermore, they pose a problem from the perspective of application developers. While these findings may not necessarily apply to other source terminologies, they highlight the conflict between preservation of authentic concept orientation and the UMLS overall desire to provide fully specified names for all source terms. The stated purpose of the UMLS [1] [2] [3] [4] is to "facilitate the development of computer systems that behave as if they "understand" the meaning of the language of biomedicine and health" [1] . The Metathesaurus (META) [2, [5] [6] [7] is "a very large… vocabulary database that contains information about biomedical and health related concepts, their various names, and the relationships among them" [2] . To achieve its purpose the UMLS integrates multiple source vocabularies into its structure. This is not an easy task as many source vocabularies overlap in their coverage, but with each source modeled differently, and oriented towards different tasks. As a result, the integration process requires certain adaptations during the incorporation of source terminologies. Creation of fully specified names, mapping of terms and synonyms to existing concepts, granularity resolution, interpretation and addition of hierarchical and lateral relationships, and assignment of semantic types of the Semantic Network [2, [8] [9] [10] are among the significant tasks in the process [11] [12] [13] [14] [15] .. Thus, despite the flexible structure of the META, these adaptations may result in a source image within the META that is not truly authentic compared to the source vocabulary. For illustration purposes we have chosen the Minimal Standard Terminology (MST) which was integrated in 2002 into the META [11] , and is one of the few UMLS source terminologies, the integration of which was reported. The MST was conceived as a "minimal" list of terms and structure to be used within computer systems to facilitate standardized reporting of gastrointestinal endoscopic examinations (GIE) [16] . It consists of 24 interlinked tables, each containing terms, qualifiers and modifiers that cover 95% of the commonly used terms in GIE reports. The MST is a well encapsulated, domain-specific terminology and therefore makes a good candidate for assessment.. We present a qualitative examination of the integration of the MST in the UMLS META with respect to the structure of the MST, the resulting knowledge representation, and the ability of the MST integration to support applications according to the original design of the MST and the original intent of the integration as stated in [11] . The integration process of the MST into the UMLS included certain adaptations that resulted in an MTHMST that is not optimally authentic compared to the source MST. In turn, this is likely to reduce the usability of the MTHMST for application developers who might want to benefit from the significant knowledge implied, but not explicitly represented in it. For reusable and authentic representation of source terminologies we propose that retaining complete concept definitions, including the exact semantic types of various concept attributes, is essential and must be preserved, even at the cost of the creation of less than fully specified terms in the UMLS. Enhanced transparency of the source vocabulary integration process will enable UMLS users to better grasp potential differences between the UMLS representation and the original source modeling. MST organization: tree structure of terms for endoscopic findings. The ERCP pancreatic component is exploded (shaded, some terms and attributes are not displayed). On top is the Pancreas section from MST On top is the Cannulation section from MST On top is part of the Diseases section from MST table 18 -Reasons for performing ERCP. The bottom section represents an effort to recreate the content of table 18 utilizing available MTHMST hierarchical and other relationships. Relationship type/attribute are marked above the arrows. Concepts are presented by preferred name (Concept ID) {semantic type}.. Indicates concepts that lack relationships to their respective diseases and attributes column content. On top is part of the Therapeutic Procedures section of MST Table 14 . The bottom section represents an effort to recreate the content of that section utilizing available MTHMST hierarchical and other relationships. Relationship type/attribute are marked above the arrows. Terms and concepts are presented separately if different than the preferred name (Term ID) ⇔ (Concept ID) {semantic type}.. Indicates concepts that lack relationships to their respective diseases and attributes column content. Table 1 List of terms used to describe observations during ERCP examination of the pancreas (MST Table 12 Each original MST term was mapped based on meaning and UMLS MTHMST source to META concepts. Overall, 1945 MST terms mapped to 1636 UMLS concepts of which, 1305 were newly introduced from the MST (80% of MST concepts). The development of electronic endoscopes and the emergence of large scale hospital information systems (HIS) in the early 1990s brought to light the need to disseminate the information collected during such examinations in a standardized, searchable and sharable manner [16] . The European Society for Gastrointestinal Endoscopy (ESGE), along with the American Society for Gastrointestinal Endoscopy (ASGE) and several other entities embarked on the creation of such standardized terminology. The resulting Minimal Standard Terminology (MST) was conceived as a "minimal" list of terms to be used within computer systems to facilitate standardized reporting of gastrointestinal endoscopic examinations (GIE). As opposed to the terminology developed by the World Organization of Digestive Endoscopy (OMED) [17, 18] , the MST is not a full fledged terminology but more of a controlled collection of preferred terms structured in a manner that follows the logical input and collection of information related to gastrointestinal endoscopy [16, 19, 20] . The MST was not designed to be an all-comprehensive collection but was meant to cover the most commonly used and widely acceptable items.. MST (Version 2) was incorporated into the UMLS January 2002 release [11] . The drive for the importation of the MST into the UMLS Metathesaurus (META) was to map MST terms into the META to provide a new machine-readable, MST-compatible terminological tool.. The UMLS META "reflects and preserves the meanings, concept names, and relationships" from its source vocabularies" [2] . Terms from various sources are mapped into concepts, and relationships between terms are mapped into relationships between the corresponding concepts. META relationships are defined as triples consisting of a relationship type (REL), a source concept, and a target concept. These mappings are based primarily on the sources, but the UMLS integration team may make adjustments as deemed appropriate. Intra-source relationships are explicit or implied in the original source vocabulary and represent its hierarchical order and cross-reference structure. Inter-source relationships are introduced by UMLS editors during content integration of a source to connect otherwise orphaned concepts or isolated components from the new source to the rich information content of other sources already existing in the META. Those relationships that are presented as hierarchical in the source are recorded as parent-child (PAR-CHD) [21] . Relationships deemed as hierarchical by the UMLS editors are recorded as broader-narrower (RB-RN) [22] . Also included are lateral (non-hierarchical) relationships. META relationships are further qualified by a label, called a relationship attribute (RELA). These RELAs are obtained from source vocabularies directly and further explain the nature of the relationship beyond the wide relationship-type categories. RELAs can be assigned to both hierarchical and lateral RELs.. The MST is organized in a set of 24 tables following four main segments: examination sites, reasons for the examination, examination specific data and organ specific information. The organ specific information is further divided into three sub-segments: findings, additional procedures and diagnoses. The MST has an overarching schema as depicted in Figure 1 (see also [16] ). However, this schema is not explicitly provided with MST's data, but rather implied through the relationship between tables and the table structure itself.. An example of a MST table is depicted in Table 1 As noted by Tringali et al [11] , the MST lacks fully specified terms but rather requires the user to post-coordinate column values based on the constraints presented by each MST table structure and content. Such organization lends itself well when used for structured or semistructured data entry or retrieval by humans or computer programs. However, the UMLS META representation of the MST is based on fully specified terms and therefore for successful yet MST-compatible integration, all pertinent post-coordination needed to occur beforehand. Hence fully specified names had to be created with appropriate assignment of Semantic Types (ST) from the UMLS Semantic Network (SN). The fully specified terms thus created in META must be linked back to their respective MST tables, attributes and attribute values. At the end of the process, the 2002 version of the UMLS META contained 1945 MST-related terms (source-abbreviated as MTHMST) with a full set of explicit relationships that can be used for interoperability between GIE clinical repositories. Below is a qualitative examination of the integration of the MST in the UMLS META. The analysis was performed using the UMLS 2007 AC release. As a reference, the published MST Version 2.0h Fixed (March 9, 1999) was used. All MST related concepts were identified based on the MST source (MTHMST). Semantic types, relationships types and relationship attributes of the relevant META concepts were identified, as were parents and children within the UMLS hierarchy.

O O O O B O O O O O O O O B O O B O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O B B O O B O O O O O O O B O O O O O B O B O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O B O O B O O O B O O O O O O O B O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O B O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O B O O B O O B O O O O O O O O B O O O O O O O O O B B O O O O O O O O O O B O O O O O O B O O O O O O B O O O B O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O B O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O B O O O O O O O B O O O O B O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O B B O O O O O O O O O O O O B O O O O B B O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O B O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O B O O B O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O B O O O O O O O B O O O O O O O O B O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O


Towards a framework for developing semantic relatedness reference standards Our objective is to develop a framework for creating reference standards for functional testing of computerized measures of semantic relatedness. Currently, research on computerized approaches to semantic relatedness between biomedical concepts relies on reference standards created for specific purposes using a variety of methods for their analysis. In most cases, these reference standards are not publicly available and the published information provided in manuscripts that evaluate computerized semantic relatedness measurement approaches is not sufficient to reproduce the results. Our proposed framework is based on the experiences of medical informatics and computational linguistics communities and addresses practical and theoretical issues with creating reference standards for semantic relatedness. We demonstrate the use of the framework on a pilot set of 101 medical term pairs rated for semantic relatedness by 13 medical coding experts. While the reliability of this particular reference standard is in the ''moderate'' range; we show that using clustering and factor analyses offers a data-driven approach to finding systematic differences among raters and identifying groups of potential outliers. We test two ontology-based measures of relatedness and provide both the reference standard containing individual ratings and the R program used to analyze the ratings as open-source. Currently, these resources are intended to be used to reproduce and compare results of studies involving computerized measures of semantic relatedness. Our framework may be extended to the development of reference standards in other research areas in medical informatics including automatic classification, information retrieval from medical records and vocabulary/ontology development. Querying electronic health records (EHR) for patients with a particular syndrome often requires using a variety of medical terms that not only denote the diagnosis itself but also symptoms and conditions closely related to the syndrome. This task is especially important for effective clinical trial recruitment and quality assurance functions where identifying these patients can assist with the process of care. For example, when searching for cases of heart failure, it is desirable to include terms for ''pulmonary edema'', ''volume/fluid overload'', ''shortness of breath,'' and possibly others to achieve the maximum sensitivity in finding relevant medical records [1] . The terms ''heart failure'', ''pulmonary edema'', ''volume/fluid overload'' and ''shortness of breath'' are clearly not synonymous but are semantically related because they denote different aspects of the same underlying condition. The terms ''cardiomegaly'' and ''splenomegaly'' may be considered semantically similar, but not necessarily related, because they share a semantic feature -both refer to an enlargement of an internal body organ. These examples illustrate that semantic relatedness is a more general notion that subsumes semantic similarity and synonymy as special cases of semantic relatedness. This hierarchical relationship between similarity and relatedness is a theoretically well established notion in lexical semantics work on general English [2, 3] that we were able to confirm experimentally in a recent psycholinguistic study in the clinical sublanguage domain. In this study, two groups of physicians were asked to rate the same 724 pairs of clinical concepts (drugs, disorders, symptoms) for the degree of either relatedness or similarity. The results of correlating the two groups' judgments clearly showed a relationship of unidirectional entailment -the pairs of terms that were judged as similar were also judged as related but not vice versa [4] .. Apart from searching EHR systems for patients with related conditions, clusters of semantically related terms are also currently used for post-marketing drug safety surveillance. In this context, drug safety reports comprising FDA Adverse Events Reporting System (AERS) may be searched using queries consisting of groups of terms (Standardized MedDRA Queries) that are indicative of the adverse event of interest [5] . For example, the adverse event of ''demyelination'' is represented by a Standardized MedDRA Query (SMQ) consisting of a total of 35 other terms including ''optic neuritis'', ''multiple sclerosis'' and ''saccadic eye movement.'' A recent study found that using SMQs for automated identification of potential adverse drug event signals in AERS yields more sensitive (albeit less specific) results than using ungrouped terms [6] . Developing tools to support querying of clinical data for drug safety surveillance is particularly important in light of the recent FDA Sentinel initiative 1 launched in May 2008 and designed to develop a national electronic system to utilize electronic health records for safety monitoring of FDA regulated medications. Electronic health records include large amounts of highly variable free text in addition to structured data and their use for drug safety surveillance will require aggregation of semantically related concepts to reduce this variability. Currently, several research groups, including ours, are investigating computerized methods for determining the strength of similarity and relatedness between medical terms and for grouping the terms based on the strength of relatedness [7] [8] [9] [10] [11] [12] . One of the critical prerequisites in these investigations is the availability of publicly accessible, validated reference standards that may be used to assess the performance of automated algorithms relative to human judgments. In previous work, we have used and made publicly available a reference standard of 30 medical term pairs [10] . These medical term pairs represent a subset of a larger set of 120 term pairs and were selected because the majority of the human annotators agreed on the strength of relatedness between the terms. In the current paper, we build on our previous work by providing a detailed analysis of the entire dataset and describing a methodology for analyzing human semantic relatedness ratings.. The main objective of this paper is to propose a framework for creating reference standards for evaluating computerized measures of semantic relatedness. Towards this objective, we validate a reference standard of medical concept pairs manually rated by professional medical coding experts for the degree of semantic relatedness between terms. We evaluate the reliability of human ratings and, based on these results, provide suggestions for future public use of this dataset to test computerized measures of semantic relatedness. We also provide both the reference standards and the statistical software for their analysis as open-source. Furthermore, we demonstrate the use of the proposed framework to evaluate two automatic computerized approaches to measuring semantic relatedness between biomedical terms. While the notions of semantic relatedness are highly subjective, particularly in a narrow terminological domain such as medicine, our work to date shows that it is possible to compile a dataset that represents a generalized notion of relatedness to be used as a reference for developing computerized tools for measuring the degree of association between medical terms. However, our work also shows that considerable disagreements may exist and a systematic approach is necessary to exploring the human ratings of similarity and relatedness. We have proposed a framework and a starting set of publicly available tools and resources to support the efforts aimed at developing reference standards for testing automated approaches to measuring semantic relatedness between medical concepts. Compiling reliable and valid reference standards is clearly a complicated process. Detailed information about the methods used to generate the dataset as well as the motivation behind the use of reliability coefficients and their interpretation should be included by study investigators in publications reporting experimental results. Without this information, the results may be uninterpretable. Our framework attempts to lay the foundation for organizing the efforts of investigators involved in research on semantic relatedness of biomedical concepts by promoting opensource tools and resources. Currently, these resources are intended to be used to reproduce and compare results of different studies on semantic relatedness; however, the framework we propose may be extended to the development of reference standards in other research areas in medical informatics including automatic classification, information retrieval from medical records and vocabulary/ ontology development. Our framework has a number of limitations that must be taken into account by anyone using the framework or the reference standards that are generated by using it. Automatic assessment of the degree to which biomedical concepts are semantically related is important in a number of different primary and secondary application contexts. The primary applications include information retrieval from clinical records and biomedical literature, and drug safety surveillance, whereas secondary applications are no less important and include support for such natural language processing tasks as automatic word sense disambiguation. In the context of drug safety surveillance, measures of semantic relatedness may be used to group together concepts that constitute an adverse effect of a medication currently defined by Standardized MedDRA Queries. Measures of semantic relatedness with subsequent clustering may enhance the process [12] reported using averaged ratings on 190 pairs of medical terms from 25 practicing primary care physicians as the reference standard. However, their corpus is limited by the fact that both the physicians and the term pairs were split for efficiency of data collection into non-overlapping groups of physician and term pair blocks effectively resulting in seven smaller (20 term pairs) datasets each annotated by at most four physicians. In creating our reference standard, we asked the raters to rely on their intuition about the terms rather than any explicit knowledge of the assertions underlying the term's meaning. The main objective of the current manuscript is to introduce a systematic approach valuable for exploring implicit semantic relatedness judgments to determine if the judgments contain an internal structure indicative of differences in conceptualization of medical knowledge. The results of this work to date show that, on the task in which the raters are given unlimited time to make the judgments, groups of raters as well as term pairs emerge with relatively high agreement even if the overall agreement for the entire group is low. We hypothesized that the disagreements may be partly due to the raters trying to define the relationship between the terms explicitly and thus resulting in variability across groups of raters. To test this hypothesis, we designed a subsequent psycholinguistic experiment [4] in which we limited the time the raters were allowed to judge each pair thus trying force them to tap into their implicit knowledge of the meanings of the terms. The results of that experiment indicated that, in this time-limited context, the physician raters did tend to agree with each other better on the majority of term pairs thus providing additional evidence for the reality of the general notion of semantic relatedness that holds between the ''core'' meanings of the medical terms.. Assessing the validity of reference standards for natural language processing and computational linguistic tasks has been a matter of continued debate in the computational linguistics community [36, 58] . This debate is highly relevant to the medical informatics community as it engages in research involving language and cognition that often requires the presence of valid reference standards. Several points of general consensus on acceptable practices in the assessment of reference standards can be identified. One of the key points relevant to the current study is the distinction between reliability and validity. According to Artstein and Poesio [58] , reliability refers to the reproducibility of the annotated reference standard -it indicates how reliable the process of the data collection is, independently of how close the human raters are to capturing the actual phenomenon (e.g., semantic relatedness between concepts). The validity, on the other hand, is said to reflect the ''trustworthiness'' of the conclusions from subsequent analyses conducted by using the reference standard.. These two notions are often treated as indistinguishable by researches; however the distinction is critical because, as clearly demonstrated by Krippendorff, and Artstein and Poesio, the interrater agreement coefficients are sensitive to factors that determine what questions can be answered by using a particular reference standard. When the research question is whether results obtained on a small manually annotated reference standard can be extrapolated to a larger dataset, one wants to be assured that if another data sample were taken from the larger dataset and manually annotated, the same (or at least similar) results would be obtained.. Hence in this scenario, the reliability of the reference standard is of utmost importance. If the research question is whether the results obtained on a reference standard with some algorithm A are the same or different from the results obtained on the same standard with algorithm B, the validity of the reference standard takes precedence. Here the researcher is concerned with whether the manual ratings on a specific reference standard accurately reflect the underlying phenomenon the raters were asked to measure. Fortunately, the differences between coefficients that are more ''sensitive'' to reliability and validity become smaller as the level of agreement and the number of raters increase. Therefore, having more than two raters for creating a reference standard is more likely to improve both the reliability and the validity of the reference standard.. Apart from selecting the right coefficient to measure reliability, choosing an appropriate interpretation of the coefficients is also a matter of debate. The medical community relies on the interpretation scales consisting of the following values: 0-0.2 -poor; 0.2-0.4 -fair; 0.4-0.6 -moderate; 0.6-0.8 -substantial; 0.8-1.0 -perfect [45, 59] . The scales proposed in computational linguistics research are much more stringent where the coefficient has to exceed 0.8 for the reference standard to be considered reliable or valid [36, 42] . One must keep in mind that these scales are established by convention and, like p-values in statistical tests, their use greatly depends on the context and the purpose for which they are being used. Also, Krippendorff's guidelines for interpreting agreement coefficients were discussed in the context of discrete nominal scales where absolute agreement on category names rather than consistency in their relative ordering is important.. The overall consistency of our dataset of 101 medical term pairs measured with average measures ICC(2,13) is in the ''perfect'' range; however the reliability measured with single measures ICC(2,1) is in the ''moderate'' range and clearly falls short of the 0.8 threshold proposed by Krippendorff and others in computational linguistics. The former coefficient is an equivalent of Cronbach's alpha and is indicative only of whether the means of the ratings in the dataset may be used as an index representative of the raters. It does not indicate that we would obtain similar results if we used a completely different set of raters. Thus, we would recommend that anyone using the entire dataset provided in Table 1 (also available as at http://rxinformatics.umn.edu) in their research ought to keep in mind that this dataset (in its entirety) is more appropriate for conducting pilot studies rather than for drawing definitive conclusions. For example, given the high consistency of the ratings, it would be appropriate to use the means for all 13 raters provided with this dataset to compare different algorithms for computing semantic relatedness with the caveat that the results of the comparison may not be used to establish the relative superiority of the algorithms in general. Any conclusions drawn from such a comparison would be limited to this particular dataset and may not be readily generalizable; however, despite this limitation, this reference standard will be useful in examining the differences between algorithms and understanding their relative strengths and weaknesses with respect to this particular dataset.. The detailed analysis of the internal structure of the medical coding experts' ratings on the medical pairs dataset suggested that the dataset contains two subgroups -one with higher inter-rater agreement than that of the entire group and one with lower agreement. The inter-rater agreement of the first group is in the ''substantial'' range (albeit still short of Krippendorff's 0.8 threshold). This agreement is calculated on eight raters, thus we propose that this subset of raters (1-3, 5, 8, 9, 10, 11) represents a reference standard with higher validity (but not reproducibility) than the set of all 13 raters.

O O O O O B I B I O O O O O O O O O B I O O O O O O O B I O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O