Drop support for Django 1.4.

Django-sortedm2m no longer supports it - and official support for 1.4 ends in 9 months as well.