From: tabo@hitech.com (Tony Abo)
Subject: Primos/Information Sector Size
Reply-To: Info-Prime@Blx-A.Prime.COM
Date: Wed, 14 Oct 1992 19:12:17 GMT


Firstly, there is no part of the PRIMOS 2k disk record that PI cannot
use.  It gets the same 2048 bytes per record that everyone gets.  2k
is not a PRIMOS restriction, however knowing how PRIMOS does its
business allows you to reduce the number of physical disk reads
required to access data.  You must realize, however that the intention
of the 2k file group is not to hold individual 2k records. (or
slightly less than 2k for PI to keep control information in the
group).  In fact there is no way prevent PI from placing two or more
records in the same group. The intention is to fill each group with
many smaller records, in the expectation that the total size of the
records and control information will not exceed the 2k group size,
thus avoiding extra disk I/O.  Some overflow is inevitable and does
not overly impact performance.  If many groups are badly overflown,
then this can impact performance, however there are techniques to deal
with these situations.

With PI dynamic files the performance issues are a little bit
different than they were in the early days.  Briefly, the main
objective in static hashed files is to keep each hashed group of
records from exceeding the PRIMOS internal record size (2k).  This is
because the hashing algorithm only tells PI's I/O routines which group
a record should be in.  Once the group is located, it then must be
searched sequentially to find the desired record.  This requires a
PRIMOS level disk read (a locate) for each 2k disk record.  In theory,
in a perfectly hashed file, only one disk read is required to find any
record in the file.  In reality, it takes a little longer, since the
group must be opened at the PRIMOS level before it can be read
(requiring at least one additional disk read per group).

The important issue with static hashed files is that the file type and
modulo be set properly to keep the records hashed evenly in the
groups, and to avoid overflown groups as much as possible.  Records
larger than 2k will always result in overflown groups, and are
therefore not well suited for static hashed files.

With dynamic files (rev 7.0 and forward, and all PI+ and PI/open
releases), things are done a little different.  When the file grows,
the modulo is automatically increased in an effort to automatically
keep the records hashed evenly and minimize overflow.  Instead of keeping
each group in a separate sequential file, the entire hashed file is
kept in one large DAM file.  The groups are now accessed as equally
sized chunks of the DAM file.  This has two implications when finding
records: 

1) a group can be read at any time without opening any additional
files since the entire hashed file is now only one PRIMOS level file.

2) When the data in a group exceeds the group size (default 2k), the
overflow will be placed in a second PRIMOS file, requiring more
complexity in PI's I/O mechanism to find the record.

Dynamic files also have a two parameters for dealing with larger
records: GROUP.SIZE and LARGE.RECORD.  These can be set in CREATE.FILE
or CONFIGURE.FILE. 

The GROUP.SIZE parameter sets the size of each group, specified in 2k
chunks.  This defaults to 1, which implies one PRIMOS disk record per
group. For files which expect to have larger records on the average,
this parameter can be used to increase the size of each chunk in the
contiguous hashed file.  If all of your records are around 2k in size,
than this parameter should be set to 2 or more.  (Remember that PI will
hash more than one record to the same group).

The LARGE.RECORD parameter is useful in files where most of the
records are relatively small, but some records are considerably
larger.  By setting this parameter (usually as a percentage of the
group size, which defaults to 80%), larger records will be removed
from the hashed file and placed in individual files, to reduce the
impact on the smaller hashed records.  This means that GROUP.SIZE does
not have to be increased to accommodate The few larger records.

In a new file with default parameters (2k groups and 80% (1638 byte)
large records), all 2k records written to the file will be placed in
separate files (where I believe there is no additional control
information so they get all of the 2k PRIMOS records.)  This would be
OK if only a few records are this size.


If all of the records are around 2k, you can set the GROUP.SIZE to 3 or 4
to ensure that two or three records can fit into each group before
overflowing. 

All of these issues are tuning issues and should not affect
application design.  If the application needs 2k records, then you
should write 2k records.  File tuning parameters can be changed later
if required to improve performance.  No application programs need to
know how the file is configured.  This applies when applications are
ported to PI/open on other hardware platforms as well.
Newsgroups: comp.sys.prime