edx/edx-platform — pull request #1499 by nparlante (2013-10-24 23:15:18+00:00)
Title: Shuffle feature for multiple choice questions

@cpennington Could you check this one out?
@rocha Note the analytics issue below
@jinpa How about you do testing, it is *your* story I believe!

Just FYI: @jaericson @shnayder

Adds studio and lms support for an option to shuffle
the displayed order of multiple choice questions.
Works by shuffling the xml tree nodes in the problem
during the get_html process. The shuffling uses the problem's seed.
The added syntax for the xml and markdown is documented in the Changelog.

One concern was: will this mess up analytics?
My experiments suggest that analytics will work fine:

The question is -- when the question choices are displayed shuffled, is enough logged so that analytics still works?

I have a 4-option problem with shuffling enabled. In this case, the options are presented in the order b a d c (i.e. 1 0 3 2). I selected option c, the last one displayed (which happens to be correct), and then looking in courseware_studentmodulehistory I have the following row

2013-10-22 22:23:40.658495|1.0|1.0|{"correct_map": {"i4x-Stanford-CS99-problem-408c0fcffb8c41ec87675d8c3f7a3b5b_2_1": {"hint": "", "hintmode": null, "correctness": "correct", "npoints": null, "msg": "", "queuestate": null}}, "input_state": {"i4x-Stanford-CS99-problem-408c0fcffb8c41ec87675d8c3f7a3b5b_2_1": {}}, "attempts": 2, "seed": 282, "done": true, "student_answers": {"i4x-Stanford-CS99-problem-408c0fcffb8c41ec87675d8c3f7a3b5b_2_1": "choice_2"}}||4|52

In student_answers I see choice_2 which is "c" in the 0-based numbering, so that's right. It looks to me that it has successfully recorded which choice I made, not getting confused by the fact that the options where displayed in a different order. The rest of the stuff in the rows looks reasonable, but mostly it's greek to me.

Because we're using standard python shuffle, knowing the seed, you can recreate the shuffle order. Maybe you never need to do this since the problem will just do it for you. Still. Noting that the logged seed above is 282:

>>> r = random.Random(282)
>>> a = [0, 1, 2, 3]
>>> r.shuffle(a)
>>> a
[1, 0, 3, 2]

Cute!