From: leichter@lrw.com (Jerry Leichter)
Newsgroups: comp.os.vms
Subject: re: SYS$LCKPAG takes forever
Date: 13 Dec 92 21:25:55 GMT
Distribution: world
Organization: The Internet


	Has anyone experienced problems locking a large sharable image into
	physical memory.

	For example, take the following Fortran BLOCK DATA declaration:

		BLOCK DATA TEST_COM

		INTEGER*4 RT_DATA(8000000)
		COMMON /TEST_COM/ RT_DATA

		END

	Link and install it as follows:

		$ LINK/NOTRACE/SHARE TEST_COM
		$ INSTALL ADD/OPEN/SHARE/WRITE directory_spec:TEST_COM.EXE

	Take the following program:

		INTEGER*4 RT_DATA(8000000)
		COMMON /TEST_COM/ RT_DATA
		INTEGER*4 SYS$LCKPAG, STATUS, INADR(2)

		INADR(1) = %LOC(RT_DATA(1))
		INADR(2) = %LOC(RT_DATA(8000000))
		STATUS = SYS$LCKPAG(INADR,,)

		END

	Link it as follows:

		$ LINK program,SYS$INPUT/OPT
		directory_spec:TEST_COM/SHARE

	The first time I run the program after installing the image, it takes 
	more than an hour to execute the SYS$LCKPAG. During subsequent runs
	it takes only a few seconds. If I link the program without linking
	to the sharable image, there is no problem.

	The first approximately 26000 pages get locked in a few seconds. Then 
	it slows down. MONITOR SYSTEM shows the program using more than 90%
	of the CPU. MONITOR MODES shows the system spending 90% of its time
	in kernel mode. MONITOR DISK shows very little I/O happening. 
	There is nothing else running on the system.

	The system is a clustered VAXstation 3200 with 64 megabytes of memory
	running VMS 5.5-2HW. I reproduced the problem on a VAXststion 4000/90.

To tell you the truth, this doesn't surprise me.  Your entire system has
131072 pages of physical memory.  You are tying to devote 62500 of them to
this one use - about half.  To lock those pages into memory, they first have
to be brought into memory, which also means free space has to be found for
them.  Once you've used up all the already-free pages - probably somewhere
around FREEGOAL pages, which is unlikely to be as high as 26000 pages, but
perhaps you just aren't using all that much of memory - that will mean
grabbing pages away from someone else.  Ultimately, it will probably mean
swapping some processes out entirely to get more space.

To make things (much) worse, at least as of VMS V4.4, SYS$LCKPAG was
implemented as a loop which acted on one page at a time.  So you make the
system go through this expensive process repeatedly, tens of thousands of
times.

It's possible that the implementation of SYS$LCKPAG changed in later versions
of VMS, but from what you are seeing, I doubt it.

The anticipated use for SYS$LCKPAG was to lock down a few pages of memory,
usually because it has to be accessed from inner-mode code that can't take
page faults.  VMS is simply not structured around a use in which half of all
free physical memory is locked down.  The fact that it survives at all is a
tribute to the robustness of the system's design.

BTW, you comment:

	If I link the program without linking to the sharable image,
	there is no problem.

I'm not sure what the implication is here.  Are you saying that when you lock
62500 pages of process private memory, you get decent response?  Or does your
test program do something else.  Of hand, I can't see why global pages should
be THAT much more expensive to lock down than process private pages - a bit
more, perhaps, because there are more data structures to update, but certainly
not a matter of seconds versus an hour.

BTW, the reason the program runs quickly the second time is that the pages
are already locked.  It still takes a couple of seconds to scan through them
all to make sure - but there's no real work for VMS to do.

							-- Jerry

Newsgroups: comp.os.vms