Only create the handle if we specify a device to gpu_init().

This should avoid initialization problems that were seen on multi-gpu clusters.