Patent Document ID: 9460135
Application ID: 14023198
Patent Flag: 1

Claim One:
1. An automated testing system comprising: one or more processors; one or more memories; and computer instructions, stored on a physical data storage device within the automated testing system that control the automated testing system to process stored test results using a robust method, which generates and uses ranks of test-result values, to identify one or more particular single-factor variants or multi-factor/multi-level variants that produce test-result distributions different from one or more other test-result distributions; for one or more of the identified variants, process stored test results to determine expected results and one of confidence intervals and significance levels, the processing to determine one of confidence intervals and significance levels including determining, by computing only a fraction of the computed values in a virtual sorted array of values computed from possible pairs of a set of initial values, a computed value stored in a particular element of the virtual sorted array associated with a particular index by ordering the set of initial values into a sorted initial-value array, and logically processing a virtual data structure, aligned with the sorted initial-value array, that includes virtual elements that contain the computed values in the virtual sorted array, by, starting with a current logical partition equal to the virtual data structure, recursively selecting a pivot element of the current logical partition, which partitions the current logical partition into a first partition, the elements of which have values less than or equal to the value of the pivot element, a second partition, the elements of which have values greater than or equal to the value of the pivot element, and a third partition, and selecting one of the first, second, or third partitions as the next current logical partition, until the current logical partition contains less than a threshold number of elements, and selecting, as the computed value at the particular index of the virtual sorted array, an element from the current logical partition; and store an indication of one or more of the one or more identified particular variants and one or more of the one of determined confidence intervals and determined significance levels in a physical data-storage device.