The bigger the virtual memory space, the bigger the adress table becomes in which is written, which virtual adress belongs to which physical adress. A big table can theoreticaly result in slower translation of the adresses and therefore in slower reading and writing speeds.