here's a big fat code contribution from Alexey Panteleev (nVidia). Failover support, overlapping computation and memory transfers, better error handling, up to 32 GPUs supported would be the most important additions... Builds on Windows, that's all I've tested soo far...