From: andrew@cubetech.com (Andrew Loewenstern)
Newsgroups: comp.sys.next.programmer
Subject: Re: Virtual memory devours my disk (SUMMARY)
Date: 10 Apr 92 05:52:27 GMT
Reply-To: andrew@cubetech.com

In article <1992Apr8.183014.1809@talus.com> ajamison@talus.com  (Alex  
Senkevitch) writes:
>  I have been working on a major SCSI project right now which presently 
>  involves VM to a great deal, and from what I understand, in 2.0, the VM page 
>  out sucks.  You'd be lucky to right 500-700k without the SOB paging out like  
>  crazy.  In 3.0, they have pushed this pageout limit to 1MB, maybe 3MB (but 
>  this is unstable).  This is not good when you are working with 100MB 
>  rasterizing jobs and such, trying to shoot them out the SCSI port (this will 
>  not work too effectively).  As far as I know, NeXT has NOT changed the VM 
>  that much.  It's still slow and it still hogs CPU time (whaaaa!).

Paging in itself is fast, but it's real speed is totally dependant on how much  
data you can blast to the disk.  Your problem is that you are just pushing your  
scsi port and devices to the absolute limit.  First you RIP part of the output  
image and separate it into RGB.  While you do that, you run out of memory and  
mach pages most of it to disk.  Then you start to blast it to the imaging  
device.  Well, eventually you'll start trying to send data that has been  
swapped to disk, and a page fault occurs, so while you are trying to blast this  
stuff out the SCSI port to the imaging device, mach is desparately trying to  
retrive the data you want from the same SCSI port and a slow disk AND it's  
trying to page out old data that you aren't going to use anymore anyway.  Well,  
mach and your software end up beating eachother senseless trying to use the  
port.  Mach gets priority, but will let your app blast more data out the scsi  
port as soon as it can get some off disk.  Effectively, the machine just grinds  
down to glacial speeds.

What you need to do is make memory transfers.  You need to structure your  
driver so it sends everything it can in real memory to the imaging device and  
then deallocates that memory.  So try to figure out how much real-memory you  
can take over without stepping on the toes of something that might want to  
access the memory.  You can probably assume that the NeXTstep apps are going to  
remain inactive the entire time you are imaging.  So allocate as big a chunk of  
page-aligned memory as you can without it going to disk (the first time you do  
it, everything else will be paged out) RIP and split, send it, then deallocate  
it.  Now there should be a nice big block of free real-memory unless something  
else on the system decides it wants memory, so allocate more and totally fill  
real-memory.  Then send that big block to the imaging device.  If you do it in  
nice big chunks like this you should get a hell of a lot more speed than having  
your app fight with mach for a single page of memory.  You also have to get rid  
of memory as soon as you don't need it or mach will page it back out to disk  
which is what you don't want.



andrew
-- 
andrew@cubetech.com
Andrew Loewenstern            |  "listen: there's a hell of a good universe
Cube Technologies, Inc.       |   next door; let's go." --- e.e. cummings
Xref: sparky comp.sys.next.programmer:3511 comp.sys.next.sysadmin:3090