Set the gem directory prefix appropriately when a specific gem binary is specified