Use pk for User since it might be customized.

Can you please release so those fixes will be available to us? We can't use the django-guardian admin as it is right now.