Set -Xms in addition to -Xmx.