From: vulture@carrion.cc.ic.ac.uk (Thomas Sippel - Dau)
Newsgroups: comp.sys.sgi
Subject: Re: Bug in SGI's malloc?
Date: 13 Oct 92 16:10:28 GMT
Reply-To: cmaae47@cc.ic.ac.uk
Organization: Imperial College of Science, Technology and Medicine

In article <5272@bacon.IMSI.COM>, jordan@IMSI.COM (Jordan Hayes) writes:
- 
- Er ... the entire process must be able to fit in the backing store, so
- malloc() should fail when swap is exhausted.

Why ? 

Is that must as in "it is technically necessary" or as in "I will not buy 
a machine that does it differently" ?

- If you had 32Mb of memory
- and 50Mb of swap, the largest process you could have would not be
- 50+32, but rather somewhat less than 50Mb (other things on the system
- are bound to be using some amount of swap space).  Ask yourself: what
- would the kernel do if a 32+50Mb process needed to be swapped out?
-
This was not even true long ago, when processes where "swapped" rather than
"paged". On a swapping system a process is either executing in memory, when
every bit of it must be in memory; idle in memory, when some bits of it
may be missing (for example being "rolled in"); or swapped out, when it is 
probably a file on disk belonging to some system user set up for this purpose.
It therefore has to be smaller than real memory, in your example, 32MB,
otherwise it could not run.

On a paging system not all of the process has to be in memory, because you 
have a memory controller that decides -on-every-access-to-memory- whether
the address you access corresponds to a memory location. If not, you have
a page fault, and the system will do something to give you the memory you
want, or decide it has had enough of you and abort you. The system also
monitors what you do with the memory you have aquired and "steals" it from 
you if you do not use it enough. It writes the memory out to disk and
adjusts the page table so that the next time you access that particular bit 
you get a page fault rather than free access.

The kernael there never needs to "swap out" a process, it executes other 
processes and steals pages as the process that has no CPU does not seem to
access memory at all (obviously).

- "Normally" ...?  Either I missed the question or you don't understand
- virtual memory.  Maximum process size is limited by swap space in
- Virtual Memory UNIX.

There are different implementations of virtual memory in unix, in fact,
the virtual memory design is one of the big four areas in which unix vendors
differentiate their systems from that of others. (The others are file 
systems, system management and administration, and sales and support)

You may have (a):

    |xxxxxxxxxxxxx|                                 <=== phys memory
    |-------------------------------------------|   <=== page disk
    |<---       virtual memory              --->|

or (b):
    |xxxxxxxxxxxxx|                                 <=== phys memory
                |-------------------------------|   <=== page disk
    |<---       virtual memory              --->|

AIX happens to use (a), IRIX (b). The advantage of (a) is that it sells 
more disk space, and that after a power failure you could restart where 
you left off, with maximally one process destroyed, if the operating system 
supports it. None of the unix systems I know of do.

The only thing you have to remember with scheme (b) is to leave a little
overlap, Irix for example always keeps a megabyte or so of real memory 
free - I don't know the exact formula.

- Remember that even if you have 96Mb of physical
- memory, if you only have 40Mb of swap you can only have 40Mb
- processes.

Again, that is true for some and not for other unix systems. Although most 
would try to discourage you quite firmly at system installation time if you 
wanted to configure less page space than physical ram.

- ps: note that this question has nothing to do with SGIs or even UNIX in
- particular.  Any VM operating system will behave similarly.

Again, I beg to disagree. I have seen many virtual memory architectures,
and they had many differences.

However, in one way you are right, your comment did not have much to do with
SGIs or Unix in general, nor even with the original posting. The question
discussed was whether processes could be fragmented in virtual memory, or
whether the (virtual) memory allocator should force them to be contiguous.

Thus if you enter a subroutine which delares a 5000x5000 local matrix, 
should you allocate 200 Mbyte, or should you wait until it gets populated,
and only allocate memory and disk space for those elements that are actually 
used. At 5000 double precision elements, each column occupies 10 4 kByte 
pages. If it is a tridiagonal matrix (as used frequently in finite element
analysis methods), it needs only 5020 pages instead of 50000 when it is dense.

The drawback is that you can overcommit the page space that you have, which
means that jobs which have already executed for days may fail near the end
because they cannot get the memory to complete. This could be because other
jobs have taken advantage of "available" page space, or because the big job
could never have completed in the first place. Usually people (users) prefer 
to take the pain when it hits them, rather than being constraint all the time
because in some (rare) cases it may actually fail.

Many architectures have the concept of memory segments, the RS/6000 for 
example has a 12 bit byte offset in a page, a 20 bit page address in the 
segmanet, and a 16 (?) bit segment number. Code, static data and program
stack live in different segments, so forcing the process to be contiguous
would mean you need more than 8 GByte of page space before you can run 
the first process, and even IBM has shied away from that. In such an 
architecture, you cannot practically force processes to be contiguous in
virtual memory.

                                     Thomas
-- 
*** This is the operative statement, all previous statements are inoperative.
*   email: cmaae47 @ cc.ic.ac.uk (Thomas Sippel - Dau) (uk.ac.ic.cc on Janet)
*   voice: +44 71 589 5111 x4937 or 4934 (day), or +44 71 823 9497 (fax)
*   snail: Imperial College of Science, Technology and Medicine
*   The Center for Computing Services, Kensington SW7 2BX, Great Britain
Newsgroups: comp.sys.sgi