Make sure we pass any set prefix path down

This is especially important for things like Qt 5, where setting and
passing the _DIR for each module would require quite a few variables
(which may vary by package being built and/or platform).

Change-Id: I3f50d672220d3a4ad60468cf4f777be909e021ef