ORDER2: A program to perform ordering-theoretic data analysis

ORDER2, a revised and expanded version of the earlier program ORDER (Bart, Lele, & Antonak, 1976), performs ordering-theoretic data analysis, a statistical method of locating the best-fitting linear or nonlinear hierarchical network among a set of item or task responses (Airasian & Bart, 1975). A prerequisite relation derived from formal logic is used to generate the hierarchy: Item i is prerequisite to Item j if and only if the response pattern (0, 1), where °represents failure on Item i and 1 represents success on Item j, occurs infrequently. That is, the response pattern (0, 1) is considered a disconfirmation that success on Item i is a prerequisite to success on Item j. To overcome the deterministic limitation shared with scalogram analysis to consider random error occurrences in the item response patterns, ordering-theoretic analysis incorporates preset tolerance levels for error. A prerequisite relation between two items is accepted only if the frequency of disconfirmatory response patterns is less than or equal to the frequency of such patterns established by the tolerance level. Detailed discussions of ordering theory, orderingtheoretic data analysis techniques, and applications to diverse data in the areas of education and psychology are contained in the texts by Krus (1975) and by Krus, Bart, and Airasian (1975). The program described in this paper incorporates the most recent refinements in the ordering-theory methodology. For example, ORDER2 allows the user to select either the standard hierarchy construction algorithm or a refined algorithm described by Bart (1978) in which item response patterns with frequencies below a specified lower bound are deleted and a hierarchy is sought with a tolerance level of O. This method allows no item intransitivities and incorporates no infrequently occurring response patterns. ORDER2 also allows the user to construct response pattern hierarchies, a technique described by Krus and Bart (1974), by which item responses are used to divide the subjects into subsets such that the response patterns for the subjects in each subset determine a linear hierarchy. More specific information about this program is contained in the ORDER2 user's manual. Input. The job deck consists of the following cards: a title card, a run parameters card, a data format card, optional scoring key(s), subject data cards, an analysis options card, tolerance level(s) card, item selection card(s), and a last card. The first card describes the problem in a text that is reproduced on the output. The run parameters include the number of subjects, items, and scoring keys (if the data are not bivalued so that 1 =success and °=failure), and unit numbers for an alternate data input medium and scratch flIes. The data format card is an I-type variable format card that indicates the location of the item responses on the following optional scoring keys, and subject responses on the subsequent data cards. Each data card must contain the data for one subject; however, the use of more than one card per subject is permitted. The first card following the data deck specifies user preference for various analyses to be performed and for printed output, and it is followed by a card that contains the tolerance level(s) selected. If the user requests the analysis of only a subset of the total item set, the next card(s) contain the numbers of the items selected. The inclusion of another analysis options card in the job deck at this point permits another analysis of the same data (e.g., a different item subset). A blank card followed by a card with the word DATA punched in Columns 1-4 permits the entry of another job deck, beginning with a title card and arranged sequentially as described above. If the last card contains the word DONE in Columns 1-4, the program terminates. Output. Optional output includes (1) matrices of item response patterns; (2) descriptive statistics, including item difficulties, score frequency distribution, scale score statistics, and the reliability indices of Cohen (1960), Ebel (I 951), and Marshall and Haertel (Note 1); (3) tables of percentages of confirmatory responses for relations of the type, "P implies Q," and/or "not P implies Q" and "P implies not Q"; (4) a summary of item hierarchies for implication relations in the form, "P is a prerequisite to Ql, Q2, Q3, ... ," using each specified tolerance level for disconfirmatory item response patterns, or, after eliminations of lowfrequency subject response patterns, a 0% tolerance level; and (5) a summary of response pattern hierarchies. Computer and Language. The program is written in FORTRAN 10 for processing by computers in the DEC-1O series and is adaptable to other systems. Well documented, the program uses mnemonic variable names to facilitate modification by the user. Restrictions. The program restricts the problem to a maximum of (1) 999 subjects, (2) 59 items (34 on the DEC-10), (3) six scoring keys, and (4) nine tolerance levels. Availability. Copies of this paper, the user's manual, and/or a source listing with test data and output may be obtained without charge from Richard F. Antonak,