Properly reconfigure the user prefs in test windows.

This is a fix for #6575.