lattice/quda — Issue #48: Multi-gpu device selection
Author: jpfoley
Created: 2012-01-31 23:12:39+00:00
State: open

Guochun and I were discussing how devices are selected when running in multi-gpu mode. At the moment devices are allocated automatically, and the lowest device numbers are selected first. This is not optimal. Take Forge, for example, where each 'node' consists of two numa nodes. Device 0 and device 1 are on the first numa node, and device 2-5 are on the second. It may be advantageous to run two jobs concurrently on each node, using two and four GPUs respectively, say. However, as it stands, those jobs would compete for device 0 and device 1, while device 4 and device 5 go unused.

We'd like to modify initQuda so that we could specify which devices to use when running in multi-gpu mode.
We could add an optional integer array argument to initQuda to specify the devices, or overload initQuda to take either an integer array or a single integer. We would keep the current setting as a default if a device array is not specified, so other code need not change.

Comments?