From: anders@lise8.lise.unit.no (Anders Christensen)
Subject: Re: Capturing System Command Output (was: SH Backquote)
Organization: /home/flipper/anders/.organization
Date: 18 Aug 92 22:32:35

In article <19920818081215SEB1525@MVS.draper.com> SEB1525@MVS.draper.com (Steve Bacher) writes:

> The recent thread on obtaining output from system commands in a
> REXX exec (aka "backquote") points up a critical need for a
> standard way of doing this.

Yes, indeed. It may be frustrating for the users, but it is (if
possible) even more frustrating for the implementors :-(

> Currently it is impossible to write an exec that can do this portably,
> even for the same platform!  There are now at least 3 different REXX
> implementations for UNIX, and they all have different conventions for
> specifying how to route command output to the stack.

There are actually four Rexx implementations available for Unix, as far
as I know.

The three different _conceptual_ methods that I am aware of are
(supposing that 'ls' is a opsys command that takes a file-spec as
parameter, which may contain wildcards, and returns all matching
filenames, one per line):

1) CALLED AS A FUNCTION  The result is a space-delimited list of the
   words and lines that the function produced:

      files = 'ls'(directory)
      say 'There are' words(files) 'files'
      do i=1 while files \= ""
         parse var files file files
         say 'file number' i 'is' file
         end

   For _non-interactive_ commands that return little output, or where
   the output is nicely structured, this seems to be a clear winner to
   me, because it is simple, clean, and easy to understand.

2) EXTRA TERM IN COMMANDS  When issuing commands, some sort of
   syntactic sugar is added that will automagically redirect the
   output (or input) to (or from) the stack. 

      'ls' directory ' >LIFO'     /* or "|rxstack" */
      say 'There are' queued() 'files'
      do i=1 while queued()>0
         parse pull file
         say 'file number' i 'is' file
         end

   Whether '>LIFO' or '|rxstack' is used is somewhat conceptually
   irrelevant for the user, even though the underlaying implementation
   might differ a lot. However, what _counts_ for the user, is that
   the same syntax is used, and that the same overall effect is
   achieved. 

   This is, of course, closely related to how it is done under CMS,
   where commands have options to flush the output to the stack, and
   where EXECIO can be used as a general purpose interface to the
   stack for operations that does not know about the stack. 

   However! Note that if a special purpose program (EXECIO or rxstack)
   is used as interface between an opsys command interpreter and the
   rexx interpreter, it will not easily scale to the situation where
   the external environment is _not_ an opsys command interpreter. To
   grab recent ftp-tread, if 'rxstack' is a program that stacks its
   input, it is possible to do:
 
       address command 
       'ls | rxstack'

   but it is not in general possible to do:

       address ftp
       'ls | rxstack'

   to list the remote files. IMHO, a rexx interpreter that is not
   capable of acting as a macro processor even for command-driven
   environments that does not know of its existence, is not a 100%
   functional interpreter.

   As far as I know, this method is the only way rexx can communicate
   with an external _interactive_ environment. 

3) READING COMMANDS WITH LINEIN.  A command is started with a builtin
   function, and the output is read using the LINEIN() function.

      rcode = popen( "ls" directory,, "ls-output" ) 
      say 'There are' lines('ls-output') 'files'
      do i=1 while lines() \= 0
         say 'file number' i 'is' linein('ls-output')
         end

   Looks like a nice extension to files (and quite similar to what is
   done in Perl ... oops, sorry, let's not talk about Perl :-)), but
   can be really eat resources, since an open process takes far more
   resources than a open file. Imagine what happens when someone opens
   30 processes in this way, and don't close them. I can also imagine
   how difficult it must be to implement this one on machines that are
   not properly multitasking!

   On the other hand, this seems like a very good way to perform
   multiple operations in the background, and it might be the right way
   to do it. But IMHO, using functions to communicate with an external
   environment is not as clear and readable as using commands (i.e
   rexx expressions that are not keywords).

Note that except from the popen() function, none of these are
_extensions_ to rexx, only a more narrow specification of what is
already stated in TRL.

No doubt, there are several conceptual additional methods ... anyone?

> The best approach available to date, hinted at by one of the posts, is
> to write a general-purpose exec that will interrogate the type of REXX
> implementation environment and do the appropriate thing. 

This is not an acceptable permanent solution, since any program is
outdated the very moment a new rexx interpreter is introduced. A
better solution might be to label the _methods_ rather than the
_interpreters_, then it would be possible to check whether an
interpreter supports e.g. the popen-based method. Then the user don't
have to update his rexxscripts to support new interpreters, but new
interpreters will support the already existing methods. 

> Is anyone working on defining a standard syntax for this capability?

Yes, probably several ... the only problem is that they don't work on
the same standard :-)  

If there anyone from the ANSI committee present, that would tell what
they intend to do with this problem?

-anders