Remove automatic loading of drm.
And makes xwl_drm_pre_init call always work

Signed-off-by: Axel Davy <axel.davy@ens.fr>