Drop support for Django 1.4 / 1.5

Should this be in this PR? Seems unrelated to dropping Django version support.