From: amylaar@opal.cs.tu-berlin.de (J"orn Rennecke)
Subject: Re: Darker Realms - The Mudlib
Organization: Techn. University of Berlin, Germany
Date: Thu, 13 Aug 1992 03:58:58 GMT

d8andjo@dtek.chalmers.se (Johan Andersson) writes:

>In article <1992Aug11.130723.22603@mcshh.Hanse.DE> amylaar@mcshh.Hanse.DE (Joern Rennecke) writes:
>>
>> Till the end of the use of lars' 3.1.2 , TubMud needed more than 90% cpu at
>> reboot time. Now its some odd 17% or 10% , with lots of idle time. Why would
>> the cpu be idling, if not because the hard disk is too slow?
>>
>A good point. This seems to argue that the lag suffered on compilation are
>due to fileaccess. I will have to check this for Genesis. The point is that
>rebooting Genesis is an order of magnitude faster when old binaries can be
>used, which is in fact most of the time. The only reason it is not even
>faster is that binaries can't be saved in objects using your switch-case code
>with string labels. This is done in a couple of preloaded objects which are
>therefore automatically forced to always be compiled.

If it would be actually worth the coding effort, this could be easily resolved
by saving the string case label list in a persistent format, and having a
table of all string case label lists in the binary.


>> Reboots are sheduled to get a new game driver version, which will typically
>> compile LPC differently, thus all old binaries are crap.

>Not so lucky here. We have 20-24 hours uptime with regular reboots. This
>is quite possibly due to not having a well tested 2.4.5 mudlib but a much
>more demanding critter. Also, there seem to be memory leaks. We have hunted
>down a few, one in your cache code, but it is tedious work.

This really astonishes me. Does the CD driver suffer so much when the mudlib
is on fire? as long as the critical master functions work, the game driver
should handle all error conditions.

>> >We have your 'global cache', what size do you use to get 88-97% hit rate?
>>
>> 1024 entries. But it's not only the cache size, but also shared string
>> propagation. (Which, as a side note, saves time & space in other places too.)
>>
>I'd say this is really due to a noncomplex 2.4.5 mudlib. Here, even if each

It's not the vanilla 2.4.5 mudlib.
The shown program sizes might be misleading, often used instructions have short
codes, some often used, formerly composite operations are now coded as single
codes, loops are rearranged to save some branches(oh, i almost forgot, jump is
gone, there are branches with 8 and 16 bit offsets instead :-) ) and struct
function has been reduced from 16 to 12 bytes.

>function call got its own slot in a 1024-entry cache it would not even
>nearly cover ~90% of the calls made. Still it is quite likely that at some
>limit the most regular calls can be contained in the cache. What that 
>limit might be is of course an interesting question worth investigating.
>
>> From your stats:
>>   The clock shows 5 days 12 hours 22 minutes 5 seconds
>>
>>   Calls to apply_low: 33709003 Cache hits: 91.15%

btw, still running, more than 7 days 3 hours,
more than 48 M calls to apply_low, 91.22% cache hit rate :-)

>Current from Genesis:
>  Local time: Wed Aug 12 13:10:37 1992
>  The game has been up since: Wed Aug 12 09:30:04 1992
>
>  Global cache, Tries: 4659277  Hits: 861816  Miss: 3797461  Rate: 18.5%
>
>I think this tells something of the difference in mudlib complexity. We

No it doesn't. the cache is rather small, and there are more malloced strings
for function calls.

>have approx 1.33Mcalls/hour whereas you have 0.25. 

The cache hit rate increases in TUBMUD when there are more players and
executed commands. Unfortunately, there are some routing problems so that
players from abroad might have more lag than on a mud in, say, sweden ot the
usa, although there IS free bandwith on some links tha can be used by means of
telnet relays.

>> You have some kind of draft version... Genesis pretended to have 3.0.51
>>
>Quite probably, but getting yours and trying to incorporate it into our GD
>is not something any of us wishes very much to do. As Jacob stated somewhere,

Wouldn't be very workable anyway. While i left 2.4.5 compatibility ( under
the assumption that the supplied master.c and simul_efun.c is used ), the
entire compiler and interpreter are heavily modified. And the simul_efun
concept is a great deal more powerful, if you liked, you could write a 
master and simul_efun set to switch between compat and 'native' mode
while the game is running.

>your code is not the easiest in the world to grasp. I wish it were, because

I have conversations with microprocessors, and they understand exactly what
i mean. Human-readable information has to stay in the naming of identifiers or
in comments for all time-critical code.

>there is a lot of what you do that we would gladly use, simply because it
>is beautiful functionality-wise, at least to my humble judgement ;-)

You can't get an omelette without breaking the eggs, and you can't program
a computer efficently without using complex algorithms where appropriate.
The size of the game driver is - compared to the data it handles - neglegible.
>
>> >  Function calls:
>> >  --------------------------
>> >  Global cache, Tries: 1525036  Hits: 312647  Miss: 1212389  Rate: 20.5%
>> >  Funmap calls: 15390
>> >  Call Cache,   Tries: 1196999  Hits: 850649  Miss: 346350  Rate: 71.1%
>>
>> >Where 'Call cache' is our 'smalltalk-like' late binding of call_others.
>>
>> >'Funmap' is for some objects where we use a hashed list for the functions. 
>>
>> This would amount to 22.7% overall miss rate. I'd like to know how you got
>> these figures, i couldn't find the profilig code in the Lpmud-CD.56.tar.Z .
>>
>It will be there in CD.57. It was something I added a month ago.
>
>> To explain something about the statistics of TubMud:
>> It starts with very high apply_low cache hit rates for the first 1 or
>> two days, like up to 98% .
>>
>This effectively means that almost all calls are in the cache. We should 
>make an experiment as to the cache size needed to accomplish that. I'm
>not entirely sure what you refer to by 'shared string propagation' but

 There are often points in the gamedriver where you get an svalue that is
 likely to be, but not necessarily is, a shared string, and where a shared
 string as result would be desirable. You can the simply test if you have
 a shared string, if yes, reuse it, if not, make one.

>all function names are shared strings and if the first thing apply_low does
>is to find the shared string equivalent of the function name called, I see

No, that's wrong. The global cache comes first. This is fine as long all the
function names are either shared strings or constant strings from the data
segment of the game driver, which are made unique in every module by gcc.

>no hitrate increase possibility there.

One of the last big sources of malloced function names was call_out. When I
changed it to use shared strings, the cache hit rate jumped up by approx.
10% of the total calls to aplly_low.
>
>Moreover, the first time a function is called in a new program. It must be
>a miss. If you, as we, have a lot of wizards coding and testing new things
>a significant number of cachemisses must occur. A 98% hitrate indicates to
>me that not much developing is going on. Might this also be a reason for
>the long uptimes? To my knowledge noone has yet fixed the fact that memory

The uptimes were deplorably short with 3.1.2 .

>is lost when compilation fails. But if there isn't much developing, there

apart from some occasional shared string, there should be nothing left now.
can't tell fro sure, though.
When the uptimes exceed a month, there will be need for a garbage collector
anyway, to be invoced every week...
This will still be IMHO the most efficient way of dealing with lost
recursive referenced arrays and mappings.

>isn't many faulty compilations and therefore not much lost memory.
>
>> The running driver uses FAST_FIT in smalloc.c
>>
>Yes, that would speed up the mud to when it is big, too. Again I wish we
>could use your code. I think Jacob took a look at it and said smalloc.c had
>grown a factor 3 since your fast-fit code was added. I recently rewrote

I've left all old code intact, even debugged some. Preprocessor directives
will pick the desired fit style. Apart from fast-fit, i also added code to
support machines where sbrk() can't be used to completely redefine malloc(),
so that you have to keep and use the old malloc(). This also added to the
source code length. moreover, i added comments everywhere where i thought
something was not obvious...
I haven't used the register directive, but have written the code with
registers in mind. It should be compiled with a high optimisation level.

>your subtract_array and intersect_array reducing the code size at least
>the same factor. The alist code is now totally gone from the CDdriver.

Ts, ts, how destructive. Mappings are not full replacement for alists.
>
>> . F_SIMUL_EFUN still uses
>> apply_low; the successor (testmuds running for some days) will use a lookup
>> table and a direct call of eval_instruction, thus it is pretty close to an
>> inherited function;
>>
>Our /secure/simul_efun is 'function mapped' so here we have a 100% 'hitrate'.

In the case of simul_efun not the best choice. Indexing with the simul_efun's
function number is faster than searching a hash table.

>I'm debating with myself if to do the same for the player files.

You could also have a binary lookup table for 2 bytes/function. But the global
cache performs so well, so why add yet another structure? :-)

	amylaar