From: engelhardt@nbivax.nbi.dk
Newsgroups: comp.compression
Subject: Re: Need Info and Advice re VMS DCX Routines
Date: 28 Aug 92 16:06:52 GMT
Organization: Niels Bohr Institute and Nordita, Copenhagen

In article <1992Aug26.132902.1@lib.haifa.ac.il>, jody@lib.haifa.ac.il (Yosef
Branse) writes:

> 1) Is it correct to assume that since the DCX routines are part of VMS, they
> will be "better" (however that may be defined - faster, more reliable, etc.)
> than other available compression packages for processing files in our system (a
> VAX 6320 running VMS 5.4-3, soon to upgrade to 5.5)?

No. They are slow (on compression) and offer poor compression. The only
advantage of the DCX$ routines is, that they allow decompression of a single
record. If you need this, and don't want to code your own stuff, you might want
to look into DCX.

> 2) Does anyone else have experience exploiting these routines? Would you
> recommend them? 

I've used them a few times. They can be handy in a few situations, but I would
not generally recommend them. But they still might do what you need.

> 3) Are there any publically available packages based on DCX?

Yes, LIBRARY comes with your favorite OS. Check
HELP LIBRARY/DATA Qualifier_Value

> 4) Assuming that the answer to 1) is yes and 3) is no - would the advantage of
> DCX be sufficient to warrant trying to master it? Or would it be best to just
> use something already available?  

Depends on what you are trying to do. If you need fast unpacking of individual
records then DCX$ might be what you want.


I have included below my sig an example (stolen from the manual) in FORTRAN on
the use of the DCX routines. If you have the FORTRAN compiler you could try
them out. They compress/decompress a file to a file. You could also try
LIBRARY/TEXT/DATA=REDUCE --- with suitable values in the /CREATE qualifier it
should give similar results.

    --- Allan.

PS: One catch: If the compressed record is longer than about 32k, DCX will not
work. The routines below will ignore these records.
-- 
Internet:     Allan.Engelhardt@nbi.dk        |  Allan Engelhardt
HEPnet:       NBIVAX::ENGELHARDT             |  Niels Bohr Institute
Bitnet/EARN:  ENGELH at CERNVM               |  Blegdamsvej 17
Phone direct: (+45)31424284<dial-tone>355    |  DK-2100 Copenhagen
Fax:          (+45)31421016                  |  DENMARK
$! ------------------ CUT HERE -----------------------
$ v='f$verify(f$trnlnm("SHARE_VERIFY"))'
$!
$! This archive created by VMS_SHARE Version 7.2-011  26-Jun-1992
$!   On 28-AUG-1992 18:00:23.94   By user ENGELHARDT (Allan Engelhardt, NBI/Aleph <Allan.Engelhardt@nbi.dk>)
$!
$! This VMS_SHARE Written by:
$!    Andy Harper, Kings College London UK
$!
$! Acknowledgements to:
$!    James Gray       - Original VMS_SHARE
$!    Michael Bednarek - Original Concept and implementation
$!
$! TO UNPACK THIS SHARE FILE, CONCATENATE ALL PARTS IN ORDER
$! AND EXECUTE AS A COMMAND PROCEDURE  (  @name  )
$!
$! THE FOLLOWING FILE(S) WILL BE CREATED AFTER UNPACKING:
$!       1. DCXCOMP.FOR;9
$!       2. DCXEXP.FOR;10
$!
$set="set"
$set symbol/scope=(nolocal,noglobal)
$f=f$parse("SHARE_TEMP","SYS$SCRATCH:.TMP_"+f$getjpi("","PID"))
$e="write sys$error  ""%UNPACK"", "
$w="write sys$output ""%UNPACK"", "
$ if f$trnlnm("SHARE_LOG") then $ w = "!"
$ ve=f$getsyi("version")
$ if ve-f$extract(0,1,ve) .ges. "4.4" then $ goto START
$ e "-E-OLDVER, Must run at least VMS 4.4"
$ v=f$verify(v)
$ exit 44
$UNPACK: SUBROUTINE ! P1=filename, P2=checksum
$ x = P1 - f$parse(P1,,,"version")
$ y = f$search(x)
$ if y .eqs. "" then $ goto file_absent
$ x = f$integer(f$parse(P1,,,"version")-";")
$ y = f$integer(f$parse(y,,,"version")-";")
$ if x .gt. y then $ goto file_absent
$ if f$mode() .eqs. "INTERACTIVE" then $ goto file_interactive
$ if x .eq. y then e "-W-EXISTS, File ''P1' exists. Skipped."
$ if x .ne. y then e "-W-NEWERVERSION, of File ''P1' exists. Skipped."
$file_delete:
$ delete 'f'*
$ exit
$file_interactive:
$ if x .eq. y then e "-W-EXISTS, File ''P1' exists."
$ if x .ne. y then e "-W-NEWERVERSION, of File ''P1' exists."
$ read/error=file_delete/end=file_delete-
  /prompt="Create new version [y/n]: " -
  sys$command x
$ if .not. x then $ e "-W-SKIPPED, File ''P1' skipped."
$ if .not. x then $ goto file_delete
$ P1 = P1 - f$parse(P1,,,"version")
$file_absent:
$ if f$parse(P1) .nes. "" then $ goto dirok
$ dn=f$parse(P1,,,"DIRECTORY")
$ w "-I-CREDIR, Creating directory ''dn'."
$ create/dir 'dn'
$ if $status then $ goto dirok
$ e "-E-CREDIRFAIL, Unable to create ''dn'. File skipped."
$ delete 'f'*
$ exit
$dirok:
$ w "-I-PROCESS, Processing file ''P1'."
$ if .not. f$verify() then $ define/user sys$output nl:
$ EDIT/TPU/NOSEC/NODIS/COM=SYS$INPUT 'f'/OUT='P1'
PROCEDURE Unpacker ON_ERROR ENDON_ERROR;SET(FACILITY_NAME,"UNPACK");SET(
SUCCESS,OFF);SET(INFORMATIONAL,OFF);f:=GET_INFO(COMMAND_LINE,"file_name");b:=
CREATE_BUFFER(f,f);p:=SPAN(" ")@r&LINE_END;POSITION(BEGINNING_OF(b));
LOOP EXITIF SEARCH(p,FORWARD)=0;POSITION(r);ERASE(r);ENDLOOP;POSITION(
BEGINNING_OF(b));g:=0;LOOP EXITIF MARK(NONE)=END_OF(b);x:=ERASE_CHARACTER(1);
IF g=0 THEN IF x="X" THEN MOVE_VERTICAL(1);ENDIF;IF x="V" THEN APPEND_LINE;
MOVE_HORIZONTAL(-CURRENT_OFFSET);MOVE_VERTICAL(1);ENDIF;IF x="+" THEN g:=1;
ERASE_LINE;ENDIF;ELSE IF x="-" THEN IF INDEX(CURRENT_LINE,"+-+-+-+-+-+-+-+")=
1 THEN g:=0;ENDIF;ENDIF;ERASE_LINE;ENDIF;ENDLOOP;t:="0123456789ABCDEF";
POSITION(BEGINNING_OF(b));LOOP r:=SEARCH("`",FORWARD);EXITIF r=0;POSITION(r);
ERASE(r);x1:=INDEX(t,ERASE_CHARACTER(1))-1;x2:=INDEX(t,ERASE_CHARACTER(1))-1;
COPY_TEXT(ASCII(16*x1+x2));ENDLOOP;WRITE_FILE(b,GET_INFO(COMMAND_LINE,
"output_file"));ENDPROCEDURE;Unpacker;QUIT;
$ delete/nolog 'f'*
$ CHECKSUM 'P1'
$ IF CHECKSUM$CHECKSUM .eqs. P2 THEN $ EXIT
$ e "-E-CHKSMFAIL, Checksum of ''P1' failed."
$ ENDSUBROUTINE
$START:
$ create 'f'
X      PROGRAM DCXCOMP
XC----------------------------------------------------------------------
XC-
XC-   Purpose and Methods : Compress a file using DCX routine.
XC-   This program is taken from the DCX manual.
XC-
XC-   Inputs  :`20
XC-   Outputs :`20
XC-   Controls:`20
XC-
XC-   Created  11-NOV-1990   Allan Engelhardt
XC   Modified :- Allan Engelhardt       2-AUG-1991  `20
XC-
XC----------------------------------------------------------------------
X      IMPLICIT NONE
X      INTEGER STATUS,IOSTAT,IO_OK,STATUS_OK
X      PARAMETER( IO_OK = 0 , STATUS_OK = 1 )
X      INCLUDE '($FORDEF)'
X      EXTERNAL DCX$_AGAIN
X      INTEGER CONTEXT
X      INTEGER MAP,MAP_LEN
X      CHARACTER*256 NORM_NAME, COMP_NAME
X      INTEGER*2 NORM_LEN,COMP_LEN
X      INTEGER NORM_LUN,COMP_LUN
X      LOGICAL EOF
X      CHARACTER*32767 RECORD,RECORD2
X      INTEGER RECORD_LEN,RECORD2_LEN
X      INTEGER GET_MAP,WRITE_MAP
X      INTEGER DCX$ANALYZE_INIT,DCX$ANALYZE_DONE,DCX$COMPRESS_INIT,
X     &   DCX$COMPRESS_DATA,DCX$COMPRESS_DONE,LIB$GET_INPUT,LIB$GET_LUN,
X     &   LIB$FREE_VM
X
X      STATUS = LIB$GET_INPUT(NORM_NAME,'File to compress: ',NORM_LEN)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      STATUS = LIB$GET_LUN(NORM_LUN)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      OPEN(UNIT=NORM_LUN,FILE=NORM_NAME(1:NORM_LEN),CARRIAGECONTROL=
X     &   'NONE',STATUS='OLD')
X      STATUS = DCX$ANALYZE_INIT(CONTEXT)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      STATUS = GET_MAP(NORM_LUN,CONTEXT,MAP,MAP_LEN)
X      DO WHILE ( STATUS.EQ.%LOC(DCX$_AGAIN) )`20
X         REWIND(UNIT=NORM_LUN)
X         STATUS = GET_MAP(NORM_LUN,CONTEXT,MAP,MAP_LEN)
X      ENDDO
X
X      WRITE(*,'(1X,A,1X,I)') 'Length of map is:',MAP_LEN
X`20
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      STATUS = DCX$ANALYZE_DONE(CONTEXT)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      REWIND(UNIT=NORM_LUN)
X      STATUS = LIB$GET_LUN(COMP_LUN)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      STATUS = LIB$GET_INPUT(COMP_NAME,'File for compressed records: ',
X     &   COMP_LEN)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      OPEN(UNIT=COMP_LUN,FILE=COMP_NAME(1:COMP_LEN),STATUS='NEW',
X     &   FORM='UNFORMATTED')
X      STATUS = DCX$COMPRESS_INIT(CONTEXT,MAP)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      CALL WRITE_MAP(COMP_LUN,%VAL(MAP),MAP_LEN)
X      EOF = .FALSE.
X      READ(UNIT=NORM_LUN,FMT='(Q,A)',IOSTAT=IOSTAT) RECORD_LEN,RECORD(1:
X     &   RECORD_LEN)
X      IF ( IOSTAT.NE.IO_OK ) THEN
X         CALL ERRSNS(,,,,STATUS)
X         IF ( STATUS.NE.FOR$_ENDDURREA ) THEN
X            CALL LIB$SIGNAL(%VAL(STATUS))
X         ELSE
X            EOF = .TRUE.
X            STATUS = STATUS_OK
X         ENDIF
X      ENDIF
X      DO WHILE ( .NOT.EOF )`20
X         STATUS = DCX$COMPRESS_DATA(CONTEXT,RECORD(1:RECORD_LEN),
X     &      RECORD2,RECORD2_LEN)
X         IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X         WRITE(UNIT=COMP_LUN) RECORD2(1:RECORD2_LEN)
X         READ(UNIT=NORM_LUN,FMT='(Q,A)',IOSTAT=IOSTAT) RECORD_LEN,
X     &      RECORD(1:RECORD_LEN)
X         IF ( IOSTAT.NE.IO_OK ) THEN
X            CALL ERRSNS(,,,,STATUS)
X            IF ( STATUS.NE.FOR$_ENDDURREA ) THEN
X               CALL LIB$SIGNAL(%VAL(STATUS))
X            ELSE
X               EOF = .TRUE.
X               STATUS = STATUS_OK
X            ENDIF
X         ENDIF
X      ENDDO
X      CLOSE(NORM_LUN)
X      CLOSE(COMP_LUN)
X      STATUS = LIB$FREE_VM(MAP_LEN,MAP)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      STATUS = DCX$COMPRESS_DONE(CONTEXT)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
XC----------------------------------------------------------------------
X      END
X
X      INTEGER FUNCTION GET_MAP(LUN,CONTEXT,MAP,MAP_LEN)
XC----------------------------------------------------------------------
XC-
XC-   Created  11-NOV-1990   Allan Engelhardt
XC-
XC----------------------------------------------------------------------
X      IMPLICIT NONE
X      INTEGER LUN,CONTEXT,MAP,MAP_LEN
X      INTEGER STATUS,IOSTAT,IO_OK,STATUS_OK
X      PARAMETER( IO_OK = 0 , STATUS_OK = 1 )
X      INCLUDE '($FORDEF)'
X      LOGICAL EOF
X      CHARACTER*32767 RECORD
X      INTEGER RECORD_LEN
X      INTEGER DCX$ANALYZE_DATA,DCX$MAKE_MAP
X
X      EOF = .FALSE.
X      READ(UNIT=LUN,FMT='(Q,A)',IOSTAT=IOSTAT) RECORD_LEN,RECORD
X      IF ( IOSTAT.NE.IO_OK ) THEN
X         CALL ERRSNS(,,,,STATUS)
X         IF ( STATUS.NE.FOR$_ENDDURREA ) THEN
X            CALL LIB$SIGNAL(%VAL(STATUS))
X         ELSE
X            EOF = .TRUE.
X            STATUS = STATUS_OK
X         ENDIF
X      ENDIF
X      DO WHILE ( .NOT.EOF )`20
X         IF ( RECORD_LEN.LT.32767/2 ) THEN
X            STATUS = DCX$ANALYZE_DATA(CONTEXT,RECORD(1:RECORD_LEN))
X         ELSE
X         ENDIF
X         STATUS = DCX$ANALYZE_DATA(CONTEXT,RECORD(1:RECORD_LEN))
X         IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X         READ(UNIT=LUN,FMT='(Q,A)',IOSTAT=IOSTAT) RECORD_LEN,RECORD
X         IF ( IOSTAT.NE.IO_OK ) THEN
X            CALL ERRSNS(,,,,STATUS)
X            IF ( STATUS.NE.FOR$_ENDDURREA ) THEN
X               CALL LIB$SIGNAL(%VAL(STATUS))
X            ELSE
X               EOF = .TRUE.
X               STATUS = STATUS_OK
X            ENDIF
X         ENDIF
X      ENDDO
X      STATUS = DCX$MAKE_MAP(CONTEXT,MAP,MAP_LEN)
X      GET_MAP = STATUS
XC----------------------------------------------------------------------
X  999 RETURN
X      END
X
X      SUBROUTINE WRITE_MAP(LUN,MAP,MAP_LEN)
XC----------------------------------------------------------------------
XC-
XC-   Created  11-NOV-1990   Allan Engelhardt
XC-
XC----------------------------------------------------------------------
X      IMPLICIT NONE
X      INTEGER LUN,MAP_LEN
X      BYTE    MAP(MAP_LEN)
X      WRITE(UNIT=LUN) MAP_LEN
X      WRITE(UNIT=LUN) MAP
XC----------------------------------------------------------------------
X  999 RETURN
X      END
$ CALL UNPACK DCXCOMP.FOR;9 1078018887
$ create 'f'
X      PROGRAM DCXEXP
XC----------------------------------------------------------------------
XC-
XC-   Purpose and Methods : Expand a compressed file using DCX$ routines.
XC-   From the DCX$ manual.
XC-
XC-   Inputs  :
XC-   Outputs :
XC-   Controls:
XC-
XC-   Created  11-NOV-1990   Allan Engelhardt
XC   Modified :- Allan Engelhardt       2-AUG-1991  `20
XC-
XC----------------------------------------------------------------------
X      IMPLICIT NONE
X      CHARACTER*256 OLD_FILE,NEW_FILE
X      INTEGER*2 OLD_LEN,NEW_LEN
X      INTEGER OLD_LUN,NEW_LUN
X      INTEGER MAP,MAP_LEN
X      EXTERNAL EXPAND_DATA
X      INTEGER LIB$GET_LUN,LIB$GET_INPUT,LIB$GET_VM,LIB$FREE_VM
X      INTEGER STATUS,IOSTAT
X
X      STATUS = LIB$GET_LUN(OLD_LUN)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      STATUS = LIB$GET_INPUT(OLD_FILE,'File to expand: ',OLD_LEN)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      OPEN(UNIT=OLD_LUN,STATUS='OLD',FILE=OLD_FILE(1:OLD_LEN),FORM=
X     &   'UNFORMATTED')
X      STATUS = LIB$GET_LUN(NEW_LUN)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      STATUS = LIB$GET_INPUT(NEW_FILE,'File to hold expanded data: ',
X     &   NEW_LEN)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      OPEN(UNIT=NEW_LUN,STATUS='NEW',FILE=NEW_FILE(1:NEW_LEN),RECL=2048,
X     &   CARRIAGECONTROL='NONE',RECORDTYPE='FIXED')
X      READ(UNIT=OLD_LUN) MAP_LEN
X      STATUS = LIB$GET_VM(MAP_LEN,MAP)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      CALL EXPAND_DATA(%VAL(MAP),MAP_LEN,OLD_LUN,NEW_LUN)
X      STATUS = LIB$FREE_VM(MAP_LEN,MAP)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
XC----------------------------------------------------------------------
X      END
X
X      SUBROUTINE EXPAND_DATA(MAP,MAP_LEN,OLD_LUN,NEW_LUN)
XC----------------------------------------------------------------------
XC-
XC-   Created  11-NOV-1990   Allan Engelhardt
XC-
XC----------------------------------------------------------------------
X      IMPLICIT NONE
X      INTEGER MAP_LEN,OLD_LUN,NEW_LUN
X      BYTE    MAP(MAP_LEN)
X      INTEGER STATUS,IOSTAT,IO_OK,STATUS_OK
X      PARAMETER( IO_OK = 0 , STATUS_OK = 1 )
X      INCLUDE '($FORDEF)'
X      INTEGER CONTEXT
X      LOGICAL EOF
X      CHARACTER*32767 RECORD,RECORD2
X      INTEGER RECORD_LEN,RECORD2_LEN
X      INTEGER DCX$EXPAND_INIT,DCX$EXPAND_DATA,DCX$EXPAND_DONE
X
X      READ(UNIT=OLD_LUN) MAP
X      STATUS = DCX$EXPAND_INIT(CONTEXT,%LOC(MAP(1)))
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X      EOF = .FALSE.
X      READ(UNIT=OLD_LUN,IOSTAT=IOSTAT) RECORD_LEN
X      IF ( IOSTAT.NE.IO_OK ) THEN
X         CALL ERRSNS(,,,,STATUS)
X         IF ( STATUS.NE.FOR$_ENDDURREA ) THEN
X            CALL LIB$SIGNAL(%VAL(STATUS))
X         ELSE
X            EOF = .TRUE.
X            STATUS = STATUS_OK
X         ENDIF
X      ENDIF
X      DO WHILE ( .NOT.EOF )
X         READ(UNIT=OLD_LUN,FMT='(Q,A)') RECORD_LEN,RECORD
X         STATUS = DCX$EXPAND_DATA(CONTEXT,RECORD(1:RECORD_LEN),RECORD2,
X     &      RECORD2_LEN)
X         IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X         WRITE(UNIT=NEW_LUN,FMT='(A)') RECORD2(1:RECORD2_LEN)
X      ENDDO
X      STATUS = DCX$EXPAND_DONE(CONTEXT)
X      IF ( .NOT.STATUS ) CALL LIB$SIGNAL(%VAL(STATUS))
X
XC----------------------------------------------------------------------
X  999 RETURN
X      END
$ CALL UNPACK DCXEXP.FOR;10 122726531
$ v=f$verify(v)
$ EXIT