The complexity is a result of the following requirements:

a) we need to support a set of providers that is not enumerable a priori;
b) each provider has a set of settings that is not enumerable a priori;
c) every deployment of open edx has a potentially different configuration requiring a different set of settings, again not enumerable a priori;
d) django's settings is basically a big global, so we need to be careful not to cruft it up with p providers * s settings * d deployments settings vars;
e) our code needs to work in test, dev, and prod, all of which have slightly different ways of configuring settings.