Multi-Objective Parallel Test-Sheet Composition Using Enhanced Particle Swarm Optimization

Introduction The key to a good test depends not only on the subjective appropriateness of test items, but also on the way the test sheet is composed. As the number of test items in an item bank is usually large, and the number of feasible combinations to form test sheets grows exponentially, it thus takes an extremely long time to set up an optimal test sheet according to the given objectives. For a formal and large-scale test, such as national certification tests or entrance examinations, the test-sheet composition problem is even more difficult to cope with, since a set of parallel test sheets needs to be composed for handling any kind of accident; for example, if some areas of the country cannot hold the test owing to inclement weather, alternative test sheets with different test item compositions but identical test ability are needed for later tests. Previous investigations have shown that a near-optimal solution for composing one test sheet is difficult to find when the number of candidate test items is larger than five thousand (Hwang, Lin, Tseng, & Lin, 2005); thus, it is much more difficult to compose a set of near-optimal parallel test sheets from a very large item bank. In this paper, we describe this problem in a general and formal way by which multiple objectives and a set of criteria can be defined by the examination committee. For instance, the test abilities of these parallel test sheets are set as close as possible while satisfying the constraints entailed by the expected testing time, the degree of discrimination, and the expected degree of relevance to each specified concept. To cope with this complex problem, an enhanced multi-objective particle swarm optimization (EMOPSO) algorithm is proposed. By employing this novel approach, the generated composition of parallel test sheets can optimize multiple objectives and meet the needs of a specified set of criteria. Background and related works Test-sheet composition Research indicates that a well-constructed test sheet not only helps in evaluating the learning achievements of students, but also facilitates the diagnosis of the problems embedded in the learning process (Hwang, 2003a, 2003b; Hwang, Tseng, & Hwang, 2008). Moreover, selecting proper test items is critical in creating a test sheet that meets multiple assessment criteria, such as the expected time needed for answering questions on the test sheet, the number of test items, the specified distribution of course concepts to be learned, and the expected discrimination or degree of difficulty (Hwang et al., 2005; Hwang, Lin, & Lin, 2006; Yin, Chang, Hwang, Hwang, & Chan, 2006). Most of the existing approaches for test-item selection focus on the composition of a single test sheet. However, for formal and large-scale tests, it is necessary to compose parallel test sheets with different sets of selected items such that the test ability (in terms of appropriate measures) of each sheet is as similar as possible. It is obvious that the problem of parallel test-sheet composition is much more difficult than that of single test-sheet composition; therefore, a more efficient and effective approach is needed for the former case. Pareto optimal solutions Single-objective optimization problems (SOOP) seek to optimize a single goal, such as cost minimization. However, multiple-objective optimization problems (MOOP), which simultaneously optimize a set of goals, are more suited to describing real-world applications. Therefore, MOOP is more computationally intensive than SOOP. A notion in MOOP is to provide the decision-maker with a set of Pareto optimal solutions, which are not dominated by any other solutions. Solution x is said to dominate another solution y if x is strictly better than y based on the objective value for at least one objective and x is no worse than y for the others. Obviously, if the final solution is not Pareto optimal, it can be improved for at least one objective without deteriorating the solution quality for other objectives. …